[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:10:58 dist-test-slave-tm1g krb5kdc[11511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:10:58 dist-test-slave-tm1g krb5kdc[11511](info): set up 2 sockets
Aug 11 03:10:58 dist-test-slave-tm1g krb5kdc[11511](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 03:10:59.501168 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:10:59 dist-test-slave-tm1g krb5kdc[11511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:10:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:10:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:10:59.700232 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:33307
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:33307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:00.047178 11528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:00.047616 11528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:00.047824 11528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:00.055258 11528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:00.055382 11528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:00.055487 11528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:00.055589 11528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:00.055686 11528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:00.066033 11528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:33307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:33307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:00.067626 11528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:00.069247 11528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:00.082219 11533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:00.082182 11535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:00.082130 11537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:00.083158 11528 server_base.cc:1034] running on GCE node
I20260811 03:11:00.084110 11528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:00.085615 11528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:00.086851 11528 hybrid_clock.cc:648] HybridClock initialized: now 1786417860086806 us; error 93 us; skew 500 ppm
I20260811 03:11:00.087272 11528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:00 dist-test-slave-tm1g krb5kdc[11511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417860, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:00.090794 11528 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:11:00.092519 11528 webserver.cc:460] Webserver started at http://127.11.60.62:32891/ using document root <none> and password file <none>
I20260811 03:11:00.093247 11528 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:00.093353 11528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:00.093606 11528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:00.097193 11528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "69356c3bf1704129bae6ffc4d02becb1"
format_stamp: "Formatted at 2026-08-11 03:11:00 on dist-test-slave-tm1g"
server_key: "1ddd850e5a0da9438ccee87d342ec03e"
server_key_iv: "4e19d9631244cb6b93824169bc311bd0"
server_key_version: "encryptionkey@0"
I20260811 03:11:00.098290 11528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "69356c3bf1704129bae6ffc4d02becb1"
format_stamp: "Formatted at 2026-08-11 03:11:00 on dist-test-slave-tm1g"
server_key: "1ddd850e5a0da9438ccee87d342ec03e"
server_key_iv: "4e19d9631244cb6b93824169bc311bd0"
server_key_version: "encryptionkey@0"
I20260811 03:11:00.104120 11528 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:11:00.107805 11545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:00.109386 11528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:11:00.109589 11528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "69356c3bf1704129bae6ffc4d02becb1"
format_stamp: "Formatted at 2026-08-11 03:11:00 on dist-test-slave-tm1g"
server_key: "1ddd850e5a0da9438ccee87d342ec03e"
server_key_iv: "4e19d9631244cb6b93824169bc311bd0"
server_key_version: "encryptionkey@0"
I20260811 03:11:00.109849 11528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:00.154345 11528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:00.159336 11528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:00.159670 11528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:00.180385 11528 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:33307
I20260811 03:11:00.180405 11606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:33307 every 8 connection(s)
I20260811 03:11:00.181934 11528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:00.185999 11607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:00.191385 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 11528
I20260811 03:11:00.191751 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:00.194525 11504 external_mini_cluster.cc:1428] Setting key 37f7af2470278369a6e4c2571e04ea14
I20260811 03:11:00.202116 11607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1: Bootstrap starting.
I20260811 03:11:00.205626 11607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:00.206779 11607 log.cc:824] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:00.209833 11607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1: No bootstrap required, opened a new log
I20260811 03:11:00.218397 11607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69356c3bf1704129bae6ffc4d02becb1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33307 } }
I20260811 03:11:00.218901 11607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:00.219024 11607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69356c3bf1704129bae6ffc4d02becb1, State: Initialized, Role: FOLLOWER
I20260811 03:11:00.219697 11607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69356c3bf1704129bae6ffc4d02becb1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33307 } }
I20260811 03:11:00.219982 11607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:00.220162 11607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:00.220391 11607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:00.222858 11607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69356c3bf1704129bae6ffc4d02becb1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33307 } }
I20260811 03:11:00.223385 11607 leader_election.cc:302] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69356c3bf1704129bae6ffc4d02becb1; no voters:
I20260811 03:11:00.224100 11607 leader_election.cc:288] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:00.224234 11614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [term 1 FOLLOWER]: Leader election won for term 1
Aug 11 03:11:00 dist-test-slave-tm1g krb5kdc[11511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:00.226712 11614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [term 1 LEADER]: Becoming Leader. State: Replica: 69356c3bf1704129bae6ffc4d02becb1, State: Running, Role: LEADER
I20260811 03:11:00.227046 11607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:00.227326 11614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69356c3bf1704129bae6ffc4d02becb1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33307 } }
I20260811 03:11:00.230885 11615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69356c3bf1704129bae6ffc4d02becb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69356c3bf1704129bae6ffc4d02becb1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33307 } } }
I20260811 03:11:00.231045 11616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69356c3bf1704129bae6ffc4d02becb1. Latest consensus state: current_term: 1 leader_uuid: "69356c3bf1704129bae6ffc4d02becb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69356c3bf1704129bae6ffc4d02becb1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33307 } } }
I20260811 03:11:00.231348 11615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:00.231451 11616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:00.232410 11621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:00.240538 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:00.202210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.11.60.62:33307)
0811 03:11:00.203169 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:00.203183 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:00.203235 (+ 52us) server_negotiation.cc:407] Connection header received
0811 03:11:00.211296 (+ 8061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:00.211318 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:00.211712 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:00.212158 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:00.215678 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:00.217413 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:00.219715 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:00.220723 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:00.220996 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:00.227887 (+ 6891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:00.227922 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:00.227938 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:00.227983 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:00.232806 (+ 4823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:00.234328 (+ 1522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:00.234341 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:00.234353 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:00.234652 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:00.235411 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:00.235419 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:00.235425 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:00.236296 (+ 871us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:00.238813 (+ 2517us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:00.239052 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 03:11:00.239348 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":403,"threads_started":1}
I20260811 03:11:00.245606 11621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:00.256157 11621 catalog_manager.cc:1347] Generated new cluster ID: f610c61b7d33488c9d14054bd4feacef
I20260811 03:11:00.256320 11621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:00.280763 11621 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:00.282812 11621 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:00.296438 11621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69356c3bf1704129bae6ffc4d02becb1: Generated new TSK 0
I20260811 03:11:00.297608 11621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:11:00.359157 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33307
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:00.714928 11636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:00.715265 11636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:00.715441 11636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:00.722831 11636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:00.722990 11636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:00.723176 11636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:00.733143 11636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33307
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:00.734458 11636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:00.735834 11636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:00.749831 11641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:00.749881 11642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:00.750041 11644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:00.751068 11636 server_base.cc:1034] running on GCE node
I20260811 03:11:00.751700 11636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:00.752652 11636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:00.753885 11636 hybrid_clock.cc:648] HybridClock initialized: now 1786417860753846 us; error 57 us; skew 500 ppm
I20260811 03:11:00.754204 11636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:00 dist-test-slave-tm1g krb5kdc[11511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417860, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:00.757467 11636 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:11:00.759168 11636 webserver.cc:460] Webserver started at http://127.11.60.1:37363/ using document root <none> and password file <none>
I20260811 03:11:00.759887 11636 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:00.760040 11636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:00.760366 11636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:00.763378 11636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "4de3e994deb84bdd95128281ce107b5c"
format_stamp: "Formatted at 2026-08-11 03:11:00 on dist-test-slave-tm1g"
server_key: "5912431d5eb96de95ac8dc6b9d314b27"
server_key_iv: "00aaed6dd33efcae7da2c88252dc3e44"
server_key_version: "encryptionkey@0"
I20260811 03:11:00.764227 11636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "4de3e994deb84bdd95128281ce107b5c"
format_stamp: "Formatted at 2026-08-11 03:11:00 on dist-test-slave-tm1g"
server_key: "5912431d5eb96de95ac8dc6b9d314b27"
server_key_iv: "00aaed6dd33efcae7da2c88252dc3e44"
server_key_version: "encryptionkey@0"
I20260811 03:11:00.769549 11636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 03:11:00.773022 11652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:00.774544 11636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:00.774730 11636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "4de3e994deb84bdd95128281ce107b5c"
format_stamp: "Formatted at 2026-08-11 03:11:00 on dist-test-slave-tm1g"
server_key: "5912431d5eb96de95ac8dc6b9d314b27"
server_key_iv: "00aaed6dd33efcae7da2c88252dc3e44"
server_key_version: "encryptionkey@0"
I20260811 03:11:00.774953 11636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:00.794955 11636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:00.799762 11636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:00.800081 11636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:00.801026 11636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:00.802426 11636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:00.802505 11636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:00.802599 11636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:00.802644 11636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:00.841306 11636 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:33521
I20260811 03:11:00.841485 11778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:33521 every 8 connection(s)
I20260811 03:11:00.842821 11636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:00.851150 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 11636
I20260811 03:11:00.851449 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:00.852113 11504 external_mini_cluster.cc:1428] Setting key 73386937749347c370e2f641b71b610d
Aug 11 03:11:00 dist-test-slave-tm1g krb5kdc[11511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417860, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:00.865162 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:00.845486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:52095 (local address 127.11.60.62:33307)
0811 03:11:00.845974 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:00.845978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:00.846982 (+ 1004us) server_negotiation.cc:407] Connection header received
0811 03:11:00.848878 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:00.848884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:00.848976 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:00.849109 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:00.850707 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:00.851579 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:00.853442 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:00.854004 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:00.854120 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:00.857872 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:00.857879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:00.857895 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:00.857962 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:00.861551 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:00.862610 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:00.862615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:00.862618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:00.862755 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:00.863285 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:00.863288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:00.863291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:00.863586 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:00.863732 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:00.864647 (+ 915us) server_negotiation.cc:299] Negotiation successful
0811 03:11:00.864839 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":280,"threads_started":1}
I20260811 03:11:00.866817 11782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:00.845987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:33307 (local address 127.11.60.1:52095)
0811 03:11:00.846814 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:00.846859 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:00.848607 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:00.849348 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:00.849356 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:00.849889 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:00.850525 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:00.850544 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:00.851748 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:00.851752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:00.853263 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:00.853280 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:00.854263 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:00.854279 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:00.854545 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:00.855293 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:00.855321 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:00.857602 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:00.861707 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:00.861720 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:00.861742 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:00.862458 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:00.862883 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:00.862891 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:00.862897 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:00.863159 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:00.863723 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:00.863734 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:00.864219 (+ 485us) client_negotiation.cc:769] Sending connection context
0811 03:11:00.864541 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 03:11:00.865174 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":327,"threads_started":1}
I20260811 03:11:00.868737 11779 heartbeater.cc:347] Connected to a master server at 127.11.60.62:33307
I20260811 03:11:00.869141 11779 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:00.870402 11779 heartbeater.cc:511] Master 127.11.60.62:33307 requested a full tablet report, sending...
I20260811 03:11:00.873086 11568 ts_manager.cc:194] Registered new tserver with Master: 4de3e994deb84bdd95128281ce107b5c (127.11.60.1:33521)
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
I20260811 03:11:00.874863 11568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:52095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:11:00.902771 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33307
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:01.301461 11787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:01.301926 11787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:01.302146 11787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:01.310020 11787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:01.310132 11787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:01.310250 11787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:01.321322 11787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33307
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:01.322795 11787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:01.324317 11787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:01.339641 11793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:01.339943 11796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:01.339568 11794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:01.340998 11787 server_base.cc:1034] running on GCE node
I20260811 03:11:01.341660 11787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:01.342885 11787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:01.344156 11787 hybrid_clock.cc:648] HybridClock initialized: now 1786417861344095 us; error 48 us; skew 500 ppm
I20260811 03:11:01.344534 11787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:01 dist-test-slave-tm1g krb5kdc[11511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:01.348100 11787 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:11:01.349769 11787 webserver.cc:460] Webserver started at http://127.11.60.2:35421/ using document root <none> and password file <none>
I20260811 03:11:01.350716 11787 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:01.350847 11787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:01.351140 11787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:01.354549 11787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "50ec47663ec94972b2aa72b10bb2b128"
format_stamp: "Formatted at 2026-08-11 03:11:01 on dist-test-slave-tm1g"
server_key: "7f91e8460f4e6001654237d842ec180e"
server_key_iv: "62c2f3b97190622551a21cb3106f1f45"
server_key_version: "encryptionkey@0"
I20260811 03:11:01.355332 11787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "50ec47663ec94972b2aa72b10bb2b128"
format_stamp: "Formatted at 2026-08-11 03:11:01 on dist-test-slave-tm1g"
server_key: "7f91e8460f4e6001654237d842ec180e"
server_key_iv: "62c2f3b97190622551a21cb3106f1f45"
server_key_version: "encryptionkey@0"
I20260811 03:11:01.361442 11787 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 03:11:01.366014 11805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:01.367528 11787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 03:11:01.367723 11787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "50ec47663ec94972b2aa72b10bb2b128"
format_stamp: "Formatted at 2026-08-11 03:11:01 on dist-test-slave-tm1g"
server_key: "7f91e8460f4e6001654237d842ec180e"
server_key_iv: "62c2f3b97190622551a21cb3106f1f45"
server_key_version: "encryptionkey@0"
I20260811 03:11:01.367978 11787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:01.390671 11787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:01.395816 11787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:01.396178 11787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:01.397487 11787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:01.399163 11787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:01.399286 11787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:01.399555 11787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:01.399703 11787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:01.448634 11787 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:44945
I20260811 03:11:01.448746 11931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:44945 every 8 connection(s)
I20260811 03:11:01.450084 11787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:01.455631 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 11787
I20260811 03:11:01.455852 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:01.456496 11504 external_mini_cluster.cc:1428] Setting key 55bbc26c25644a2b4f681df268c63224
Aug 11 03:11:01 dist-test-slave-tm1g krb5kdc[11511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
I20260811 03:11:01.481614 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:01.453019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:51969 (local address 127.11.60.62:33307)
0811 03:11:01.453506 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:01.453511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:01.454906 (+ 1395us) server_negotiation.cc:407] Connection header received
0811 03:11:01.457857 (+ 2951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:01.457863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:01.457960 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:01.458081 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:01.460321 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:01.461107 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:01.464547 (+ 3440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:01.464993 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:01.465101 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:01.473164 (+ 8063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:01.473174 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:01.473181 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:01.473237 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:01.477666 (+ 4429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:01.478833 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:01.478838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:01.478840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:01.479148 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:01.479673 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:01.479678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:01.479680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:01.480026 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:01.480191 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:01.480972 (+ 781us) server_negotiation.cc:299] Negotiation successful
0811 03:11:01.481221 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":251,"threads_started":1}
I20260811 03:11:01.482733 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:01.453666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:33307 (local address 127.11.60.2:51969)
0811 03:11:01.454731 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:01.454779 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:01.457411 (+ 2632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:01.458366 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:01.458384 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:01.459197 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:01.460103 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:01.460132 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:01.461307 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:01.461320 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:01.464351 (+ 3031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:01.464368 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:01.465251 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:01.465261 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:01.465668 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:01.469779 (+ 4111us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:01.469810 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:01.472776 (+ 2966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:01.477946 (+ 5170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:01.477954 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:01.477970 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:01.478585 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:01.479280 (+ 695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:01.479285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:01.479288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:01.479536 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:01.480162 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:01.480171 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:01.480563 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 03:11:01.480849 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 03:11:01.481450 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":458,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
I20260811 03:11:01.484413 11932 heartbeater.cc:347] Connected to a master server at 127.11.60.62:33307
I20260811 03:11:01.484764 11932 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:01.485556 11932 heartbeater.cc:511] Master 127.11.60.62:33307 requested a full tablet report, sending...
I20260811 03:11:01.487285 11568 ts_manager.cc:194] Registered new tserver with Master: 50ec47663ec94972b2aa72b10bb2b128 (127.11.60.2:44945)
I20260811 03:11:01.488183 11568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:51969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:11:01.510870 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33307
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:11:01.878254 11779 heartbeater.cc:503] Master 127.11.60.62:33307 was elected leader, sending a full tablet report...
W20260811 03:11:01.934656 11938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:01.934937 11938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:01.935105 11938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:01.942641 11938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:01.942747 11938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:01.942945 11938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:01.953596 11938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33307
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:01.955013 11938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:01.956684 11938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:01.971247 11944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:01.971367 11945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:01.971333 11948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:01.972456 11938 server_base.cc:1034] running on GCE node
I20260811 03:11:01.973199 11938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:01.974190 11938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:01.975405 11938 hybrid_clock.cc:648] HybridClock initialized: now 1786417861975324 us; error 63 us; skew 500 ppm
I20260811 03:11:01.975741 11938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:01 dist-test-slave-tm1g krb5kdc[11511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:01.979174 11938 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:11:01.980861 11938 webserver.cc:460] Webserver started at http://127.11.60.3:41751/ using document root <none> and password file <none>
I20260811 03:11:01.981581 11938 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:01.981691 11938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:01.981976 11938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:01.985200 11938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "5c71563930364b2d9e9512a21bda9a7a"
format_stamp: "Formatted at 2026-08-11 03:11:01 on dist-test-slave-tm1g"
server_key: "bca6923ed41cabb83ea4a36c425d4606"
server_key_iv: "c4fe5ca9990371e612ba033ac07ab048"
server_key_version: "encryptionkey@0"
I20260811 03:11:01.986054 11938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "5c71563930364b2d9e9512a21bda9a7a"
format_stamp: "Formatted at 2026-08-11 03:11:01 on dist-test-slave-tm1g"
server_key: "bca6923ed41cabb83ea4a36c425d4606"
server_key_iv: "c4fe5ca9990371e612ba033ac07ab048"
server_key_version: "encryptionkey@0"
I20260811 03:11:01.991518 11938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 03:11:01.995488 11955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:01.996855 11938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:11:01.997053 11938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "5c71563930364b2d9e9512a21bda9a7a"
format_stamp: "Formatted at 2026-08-11 03:11:01 on dist-test-slave-tm1g"
server_key: "bca6923ed41cabb83ea4a36c425d4606"
server_key_iv: "c4fe5ca9990371e612ba033ac07ab048"
server_key_version: "encryptionkey@0"
I20260811 03:11:01.997284 11938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:02.040434 11938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:02.045405 11938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:02.045758 11938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:02.046934 11938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:02.048465 11938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:02.048635 11938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:02.048761 11938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:02.048810 11938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:02.098628 11938 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:45357
I20260811 03:11:02.098702 12084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:45357 every 8 connection(s)
I20260811 03:11:02.099982 11938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:02.107044 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 11938
I20260811 03:11:02.107256 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:02.107728 11504 external_mini_cluster.cc:1428] Setting key 968cb814fe368192148e894668776c2c
Aug 11 03:11:02 dist-test-slave-tm1g krb5kdc[11511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:02.122593 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:02.102845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:53027 (local address 127.11.60.62:33307)
0811 03:11:02.103315 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:02.103319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:02.104315 (+ 996us) server_negotiation.cc:407] Connection header received
0811 03:11:02.106236 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:02.106241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:02.106357 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:02.106552 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:02.108433 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.109320 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.111599 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.112217 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.112333 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:02.116262 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:02.116267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:02.116270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:02.116303 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:02.119391 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.120131 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.120137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.120141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.120282 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.120723 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.120729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.120732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.121019 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:02.121184 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:02.121888 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 03:11:02.122118 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":225,"threads_started":1}
I20260811 03:11:02.123979 12088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:02.103417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:33307 (local address 127.11.60.3:53027)
0811 03:11:02.104142 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:02.104186 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:02.105938 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:02.106791 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:02.106800 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:02.107535 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:02.108241 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:02.108261 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.109483 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.109489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:02.111423 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:02.111438 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.112421 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.112425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:02.112691 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:02.113485 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:02.113508 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:02.116051 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:02.119540 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:02.119549 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:02.119565 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:02.120005 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:02.120402 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:02.120407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:02.120410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:02.120602 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:02.121152 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:02.121161 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:02.121509 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 03:11:02.121783 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 03:11:02.122443 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":279,"threads_started":1}
I20260811 03:11:02.125389 12085 heartbeater.cc:347] Connected to a master server at 127.11.60.62:33307
I20260811 03:11:02.125671 12085 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:02.126617 12085 heartbeater.cc:511] Master 127.11.60.62:33307 requested a full tablet report, sending...
I20260811 03:11:02.128389 11568 ts_manager.cc:194] Registered new tserver with Master: 5c71563930364b2d9e9512a21bda9a7a (127.11.60.3:45357)
I20260811 03:11:02.129583 11568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:53027
I20260811 03:11:02.137061 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 03:11:02 dist-test-slave-tm1g krb5kdc[11511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417862, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 03:11:02 dist-test-slave-tm1g krb5kdc[11511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417862, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:02.196807 12093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:02.179073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56866 (local address 127.11.60.1:33521)
0811 03:11:02.179867 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:02.179892 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:02.180054 (+ 162us) server_negotiation.cc:407] Connection header received
0811 03:11:02.180297 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:02.180306 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:02.180581 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:02.180762 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:02.182505 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.183679 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.185208 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.185830 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.185981 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:02.190843 (+ 4862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:02.190856 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:02.190872 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:02.190914 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:02.193947 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.194852 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.194860 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.194862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.195003 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.195391 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.195395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.195397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.195744 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:02.195984 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:02.196263 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 03:11:02.196476 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":400,"threads_started":1}
W20260811 03:11:02.198262 11686 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56866
Aug 11 03:11:02 dist-test-slave-tm1g krb5kdc[11511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 03:11:02 dist-test-slave-tm1g krb5kdc[11511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:02.251583 12093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:02.236300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56876 (local address 127.11.60.1:33521)
0811 03:11:02.236568 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:02.236578 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:02.237047 (+ 469us) server_negotiation.cc:407] Connection header received
0811 03:11:02.237280 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:02.237285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:02.237390 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:02.237507 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:02.239274 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.240010 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.241268 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.241709 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.241856 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:02.246024 (+ 4168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:02.246029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:02.246032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:02.246068 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:02.248895 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.249590 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.249594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.249596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.249756 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.250157 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.250160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.250162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.250549 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:02.250746 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:02.251138 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 03:11:02.251316 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 03:11:02.253562 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 11636
I20260811 03:11:02.267417 12093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:02.254699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.11.60.1:33521)
0811 03:11:02.254921 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:02.254929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:02.255221 (+ 292us) server_negotiation.cc:407] Connection header received
0811 03:11:02.255354 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:02.255357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:02.255421 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:02.255501 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:02.256813 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.257554 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.259145 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.259553 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.259645 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:02.262025 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:02.262029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:02.262032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:02.262058 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:02.264769 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.265446 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.265450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.265453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.265614 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.265962 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.265965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.265967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.266268 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:02.266640 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:02.266842 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 03:11:02.267004 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 03:11:02.268204 11774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:02.354022 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 11636
I20260811 03:11:02.373144 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 11787
Aug 11 03:11:02 dist-test-slave-tm1g krb5kdc[11511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:11:02.390643 12102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:02.374050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44032 (local address 127.11.60.2:44945)
0811 03:11:02.374712 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:02.374720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:02.374747 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:11:02.374809 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:02.374815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:02.375029 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:02.375159 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:02.376337 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.377364 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.378865 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.379417 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.379538 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:02.383789 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:02.383807 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:02.383826 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:02.383884 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:02.387737 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.388438 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.388447 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.388449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.388605 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.388974 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.388977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.388979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.389354 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:02.389629 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:02.389991 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 03:11:02.390223 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":372,"threads_started":1}
I20260811 03:11:02.392115 11927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:02.476239 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 11787
I20260811 03:11:02.492334 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 11938
Aug 11 03:11:02 dist-test-slave-tm1g krb5kdc[11511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:11:02.510030 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:02.493502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46262 (local address 127.11.60.3:45357)
0811 03:11:02.494187 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:02.494194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:02.494252 (+ 58us) server_negotiation.cc:407] Connection header received
0811 03:11:02.494367 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:02.494434 (+ 67us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:02.494719 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:02.494921 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:02.496210 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.497255 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.498823 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:02.499460 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:02.499572 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:02.504110 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:02.504124 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:02.504139 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:02.504176 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:02.507180 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.507914 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.507923 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.507927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.508099 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:02.508503 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:02.508507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:02.508509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:02.508934 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:02.509203 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:02.509504 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 03:11:02.509718 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":354,"threads_started":1}
I20260811 03:11:02.511603 12078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:02.596225 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 11938
I20260811 03:11:02.615520 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 11528
I20260811 03:11:02.616804 11602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:02.721352 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 11528
2026-08-11T03:11:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4409 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:11:02 dist-test-slave-tm1g krb5kdc[12110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:11:02 dist-test-slave-tm1g krb5kdc[12110](info): set up 2 sockets
Aug 11 03:11:02 dist-test-slave-tm1g krb5kdc[12110](info): commencing operation
krb5kdc: starting...
W20260811 03:11:03.886039 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.006s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:11:04 dist-test-slave-tm1g krb5kdc[12110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:11:04.090797 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:46723
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:36729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:46723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:04.422797 12124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:04.423233 12124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:04.423382 12124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:04.430797 12124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:04.430902 12124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:04.430981 12124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:04.431074 12124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:04.431161 12124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:04.441053 12124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:36729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:46723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:46723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:04.442440 12124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:04.443770 12124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:04.456393 12129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:04.456358 12130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:04.456707 12132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:04.457072 12124 server_base.cc:1034] running on GCE node
I20260811 03:11:04.457923 12124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:04.459471 12124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:04.460742 12124 hybrid_clock.cc:648] HybridClock initialized: now 1786417864460691 us; error 61 us; skew 500 ppm
I20260811 03:11:04.461108 12124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:04 dist-test-slave-tm1g krb5kdc[12110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417864, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:04.468878 12124 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:11:04.470543 12124 webserver.cc:460] Webserver started at http://127.11.60.62:42857/ using document root <none> and password file <none>
I20260811 03:11:04.471290 12124 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:04.471436 12124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:04.471763 12124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:04.474915 12124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "fbf3b8709b3645ac9e5f73864efd44a6"
format_stamp: "Formatted at 2026-08-11 03:11:04 on dist-test-slave-tm1g"
server_key: "81037444abfd0205ba869dcccc3c1278"
server_key_iv: "030baa3399221feda6808dbed0982809"
server_key_version: "encryptionkey@0"
I20260811 03:11:04.475693 12124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "fbf3b8709b3645ac9e5f73864efd44a6"
format_stamp: "Formatted at 2026-08-11 03:11:04 on dist-test-slave-tm1g"
server_key: "81037444abfd0205ba869dcccc3c1278"
server_key_iv: "030baa3399221feda6808dbed0982809"
server_key_version: "encryptionkey@0"
I20260811 03:11:04.480973 12124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:11:04.484942 12139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:04.486510 12124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:04.486704 12124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "fbf3b8709b3645ac9e5f73864efd44a6"
format_stamp: "Formatted at 2026-08-11 03:11:04 on dist-test-slave-tm1g"
server_key: "81037444abfd0205ba869dcccc3c1278"
server_key_iv: "030baa3399221feda6808dbed0982809"
server_key_version: "encryptionkey@0"
I20260811 03:11:04.486919 12124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:04.514642 12124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:04.519171 12124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:04.519506 12124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:04.539613 12124 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:46723
I20260811 03:11:04.539638 12197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:46723 every 8 connection(s)
I20260811 03:11:04.541066 12124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:04.544984 12198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:04.551242 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 12124
I20260811 03:11:04.551496 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:04.552340 11504 external_mini_cluster.cc:1428] Setting key ab295e6e81d7282f90acb7e6e6163852
I20260811 03:11:04.556478 12198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6: Bootstrap starting.
I20260811 03:11:04.560318 12198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:04.561452 12198 log.cc:824] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:04.564710 12198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6: No bootstrap required, opened a new log
Aug 11 03:11:04 dist-test-slave-tm1g krb5kdc[12110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:04.575618 12198 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf3b8709b3645ac9e5f73864efd44a6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46723 } }
I20260811 03:11:04.576108 12198 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:04.576225 12198 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbf3b8709b3645ac9e5f73864efd44a6, State: Initialized, Role: FOLLOWER
I20260811 03:11:04.576907 12198 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf3b8709b3645ac9e5f73864efd44a6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46723 } }
I20260811 03:11:04.577219 12198 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:04.577375 12198 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:04.577586 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:04.554588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.11.60.62:46723)
0811 03:11:04.555381 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:04.555394 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:04.555433 (+ 39us) server_negotiation.cc:407] Connection header received
0811 03:11:04.557182 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:04.557203 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:04.557572 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:04.557882 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:04.559890 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:04.561295 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:04.562958 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:04.563629 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:04.563760 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:04.568615 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:04.568625 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:04.568645 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:04.568693 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:04.572871 (+ 4178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:04.573573 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:04.573583 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:04.573592 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:04.573842 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:04.574191 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:04.574196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:04.574199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:04.574970 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:04.575273 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:04.575682 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 03:11:04.576164 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":370,"threads_started":1}
I20260811 03:11:04.577587 12198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:04.580147 12198 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf3b8709b3645ac9e5f73864efd44a6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46723 } }
I20260811 03:11:04.580669 12198 leader_election.cc:302] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbf3b8709b3645ac9e5f73864efd44a6; no voters:
I20260811 03:11:04.581266 12198 leader_election.cc:288] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:04.581403 12204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:04.582158 12204 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [term 1 LEADER]: Becoming Leader. State: Replica: fbf3b8709b3645ac9e5f73864efd44a6, State: Running, Role: LEADER
I20260811 03:11:04.582803 12204 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf3b8709b3645ac9e5f73864efd44a6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46723 } }
I20260811 03:11:04.583422 12198 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:04.585769 12205 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbf3b8709b3645ac9e5f73864efd44a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf3b8709b3645ac9e5f73864efd44a6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46723 } } }
I20260811 03:11:04.586098 12205 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:04.588447 12210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:04.588292 12206 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbf3b8709b3645ac9e5f73864efd44a6. Latest consensus state: current_term: 1 leader_uuid: "fbf3b8709b3645ac9e5f73864efd44a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf3b8709b3645ac9e5f73864efd44a6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46723 } } }
I20260811 03:11:04.588739 12206 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:04.593688 12210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:04.604030 12210 catalog_manager.cc:1347] Generated new cluster ID: b0856b54ba954cada5f4d92b4fa6e6c3
I20260811 03:11:04.604156 12210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:04.610925 12210 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:04.612489 12210 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:04.625792 12210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbf3b8709b3645ac9e5f73864efd44a6: Generated new TSK 0
I20260811 03:11:04.626910 12210 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:11:04.681605 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:46723
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:36729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:05.021420 12226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:05.021716 12226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:05.021888 12226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:05.029908 12226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:05.030030 12226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:05.030218 12226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:05.040042 12226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:36729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:46723
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:05.041352 12226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:05.042686 12226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:05.056221 12233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:05.056165 12239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:05.056280 12234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:05.062485 12226 server_base.cc:1034] running on GCE node
I20260811 03:11:05.063143 12226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:05.064105 12226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:05.065478 12226 hybrid_clock.cc:648] HybridClock initialized: now 1786417865065386 us; error 78 us; skew 500 ppm
I20260811 03:11:05.065835 12226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:05 dist-test-slave-tm1g krb5kdc[12110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417865, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:05.069479 12226 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:11:05.071316 12226 webserver.cc:460] Webserver started at http://127.11.60.1:37789/ using document root <none> and password file <none>
I20260811 03:11:05.072090 12226 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:05.072220 12226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:05.072566 12226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:05.075656 12226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "449f757ebcc346ae9e4739b765a7598c"
format_stamp: "Formatted at 2026-08-11 03:11:05 on dist-test-slave-tm1g"
server_key: "d051f8dbbd48b3460dbf1f06a06f01b3"
server_key_iv: "3f055bb4be8c49f922c15619856b197d"
server_key_version: "encryptionkey@0"
I20260811 03:11:05.076473 12226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "449f757ebcc346ae9e4739b765a7598c"
format_stamp: "Formatted at 2026-08-11 03:11:05 on dist-test-slave-tm1g"
server_key: "d051f8dbbd48b3460dbf1f06a06f01b3"
server_key_iv: "3f055bb4be8c49f922c15619856b197d"
server_key_version: "encryptionkey@0"
I20260811 03:11:05.081882 12226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 03:11:05.085631 12247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:05.087060 12226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:05.087241 12226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "449f757ebcc346ae9e4739b765a7598c"
format_stamp: "Formatted at 2026-08-11 03:11:05 on dist-test-slave-tm1g"
server_key: "d051f8dbbd48b3460dbf1f06a06f01b3"
server_key_iv: "3f055bb4be8c49f922c15619856b197d"
server_key_version: "encryptionkey@0"
I20260811 03:11:05.087457 12226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:05.118216 12226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:05.122467 12226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:05.122772 12226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:05.123773 12226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:05.125243 12226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:05.125362 12226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:05.125519 12226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:05.125592 12226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:05.163326 12226 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:39873
I20260811 03:11:05.163375 12365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:39873 every 8 connection(s)
I20260811 03:11:05.164655 12226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:05.173193 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 12226
I20260811 03:11:05.173460 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:05.174186 11504 external_mini_cluster.cc:1428] Setting key fa7bd2f19762996c2795352c8a452b99
Aug 11 03:11:05 dist-test-slave-tm1g krb5kdc[12110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417865, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:05.186878 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:05.167379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:41965 (local address 127.11.60.62:46723)
0811 03:11:05.167889 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:05.167896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:05.168821 (+ 925us) server_negotiation.cc:407] Connection header received
0811 03:11:05.170756 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:05.170762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:05.170833 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:05.170932 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:05.172554 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:05.173504 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:05.175548 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:05.176080 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:05.176225 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:05.180648 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:05.180654 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:05.180658 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:05.180693 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:05.183420 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:05.184298 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:05.184303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:05.184305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:05.184430 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:05.184959 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:05.184966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:05.184975 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:05.185247 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:05.185429 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:05.186195 (+ 766us) server_negotiation.cc:299] Negotiation successful
0811 03:11:05.186490 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":239,"threads_started":1}
I20260811 03:11:05.187978 12372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:05.167950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:46723 (local address 127.11.60.1:41965)
0811 03:11:05.168657 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:05.168696 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:05.170463 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:05.171162 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:05.171170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:05.171697 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:05.172322 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:05.172342 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:05.173721 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:05.173725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:05.175385 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:05.175398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:05.176342 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:05.176350 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:05.176656 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:05.177431 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:05.177465 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:05.180386 (+ 2921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:05.183643 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:05.183655 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:05.183674 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:05.184150 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:05.184557 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:05.184562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:05.184566 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:05.184832 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:05.185401 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:05.185410 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:05.185790 (+ 380us) client_negotiation.cc:769] Sending connection context
0811 03:11:05.186070 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 03:11:05.186684 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":272,"threads_started":1}
I20260811 03:11:05.189754 12366 heartbeater.cc:347] Connected to a master server at 127.11.60.62:46723
I20260811 03:11:05.190094 12366 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:05.191054 12366 heartbeater.cc:511] Master 127.11.60.62:46723 requested a full tablet report, sending...
I20260811 03:11:05.193689 12160 ts_manager.cc:194] Registered new tserver with Master: 449f757ebcc346ae9e4739b765a7598c (127.11.60.1:39873)
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
I20260811 03:11:05.195782 12160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:41965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:11:05.226002 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:46723
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:36729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:05.568946 12375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:05.569242 12375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:05.569418 12375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:05.576982 12375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:05.577096 12375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:05.577298 12375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:05.587706 12375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:36729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:46723
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:05.589201 12375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:05.590677 12375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:05.604696 12380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:05.604787 12383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:05.604800 12381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:05.605894 12375 server_base.cc:1034] running on GCE node
I20260811 03:11:05.606564 12375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:05.607481 12375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:05.608729 12375 hybrid_clock.cc:648] HybridClock initialized: now 1786417865608694 us; error 46 us; skew 500 ppm
I20260811 03:11:05.609086 12375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:05 dist-test-slave-tm1g krb5kdc[12110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417865, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:05.612322 12375 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:11:05.613831 12375 webserver.cc:460] Webserver started at http://127.11.60.2:45243/ using document root <none> and password file <none>
I20260811 03:11:05.614653 12375 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:05.614801 12375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:05.615129 12375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:05.618067 12375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "cea7f9be783046acb95bcdbad425ec76"
format_stamp: "Formatted at 2026-08-11 03:11:05 on dist-test-slave-tm1g"
server_key: "4445b579576c409aed4f01b8d19de904"
server_key_iv: "5cecc562b17576f5f2561f3e349997de"
server_key_version: "encryptionkey@0"
I20260811 03:11:05.618937 12375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "cea7f9be783046acb95bcdbad425ec76"
format_stamp: "Formatted at 2026-08-11 03:11:05 on dist-test-slave-tm1g"
server_key: "4445b579576c409aed4f01b8d19de904"
server_key_iv: "5cecc562b17576f5f2561f3e349997de"
server_key_version: "encryptionkey@0"
I20260811 03:11:05.624148 12375 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:11:05.627894 12391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:05.629241 12375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:11:05.629434 12375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "cea7f9be783046acb95bcdbad425ec76"
format_stamp: "Formatted at 2026-08-11 03:11:05 on dist-test-slave-tm1g"
server_key: "4445b579576c409aed4f01b8d19de904"
server_key_iv: "5cecc562b17576f5f2561f3e349997de"
server_key_version: "encryptionkey@0"
I20260811 03:11:05.629657 12375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:05.653645 12375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:05.658346 12375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:05.658658 12375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:05.659713 12375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:05.661111 12375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:05.661229 12375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:05.661386 12375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:05.661459 12375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:05.697633 12375 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:36287
I20260811 03:11:05.697723 12515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:36287 every 8 connection(s)
I20260811 03:11:05.699054 12375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:05.707340 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 12375
I20260811 03:11:05.707578 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:05.708194 11504 external_mini_cluster.cc:1428] Setting key 6e6f9f537d466ab0c7652b92fbb7c32e
Aug 11 03:11:05 dist-test-slave-tm1g krb5kdc[12110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417865, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:05.722664 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:05.702041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:36277 (local address 127.11.60.62:46723)
0811 03:11:05.702656 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:05.702661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:05.703605 (+ 944us) server_negotiation.cc:407] Connection header received
0811 03:11:05.705709 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:05.705716 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:05.705817 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:05.705959 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:05.708061 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:05.708895 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:05.710922 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:05.711708 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:05.711872 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:05.715571 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:05.715580 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:05.715589 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:05.715639 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:05.719040 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:05.720033 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:05.720037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:05.720040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:05.720177 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:05.720666 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:05.720670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:05.720672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:05.720969 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:05.721089 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:05.722065 (+ 976us) server_negotiation.cc:299] Negotiation successful
0811 03:11:05.722249 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":380,"threads_started":1}
I20260811 03:11:05.724120 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:05.702558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:46723 (local address 127.11.60.2:36277)
0811 03:11:05.703422 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:05.703465 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:05.705399 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:05.706214 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:05.706222 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:05.706880 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:05.707828 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:05.707857 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:05.709074 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:05.709078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:05.710728 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:05.710741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:05.711965 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:05.711971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:05.712182 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:05.712988 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:05.713012 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:05.715307 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:05.719190 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:05.719202 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:05.719223 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:05.719892 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:05.720290 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:05.720297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:05.720303 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:05.720551 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:05.721105 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:05.721116 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:05.721604 (+ 488us) client_negotiation.cc:769] Sending connection context
0811 03:11:05.721952 (+ 348us) client_negotiation.cc:240] Negotiation successful
0811 03:11:05.722568 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":334,"threads_started":1}
I20260811 03:11:05.725752 12517 heartbeater.cc:347] Connected to a master server at 127.11.60.62:46723
I20260811 03:11:05.726217 12517 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:05.727015 12517 heartbeater.cc:511] Master 127.11.60.62:46723 requested a full tablet report, sending...
I20260811 03:11:05.728621 12160 ts_manager.cc:194] Registered new tserver with Master: cea7f9be783046acb95bcdbad425ec76 (127.11.60.2:36287)
I20260811 03:11:05.729494 12160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:36277
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:11:05.761794 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:46723
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:36729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:06.101138 12524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:06.101491 12524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:06.101671 12524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:06.108897 12524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:06.109056 12524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:06.109247 12524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:06.119215 12524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:36729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:46723
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:06.120705 12524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:06.122064 12524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:06.136081 12531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:06.136020 12533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:06.136022 12535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:06.137209 12524 server_base.cc:1034] running on GCE node
I20260811 03:11:06.137878 12524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:06.138957 12524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:06.140178 12524 hybrid_clock.cc:648] HybridClock initialized: now 1786417866140117 us; error 39 us; skew 500 ppm
I20260811 03:11:06.140508 12524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:06 dist-test-slave-tm1g krb5kdc[12110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417866, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:06.144851 12524 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:11:06.146430 12524 webserver.cc:460] Webserver started at http://127.11.60.3:37273/ using document root <none> and password file <none>
I20260811 03:11:06.147150 12524 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:06.147256 12524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:06.147531 12524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:06.150628 12524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "2e59479f2b704c5f87ec22c5b0c2c489"
format_stamp: "Formatted at 2026-08-11 03:11:06 on dist-test-slave-tm1g"
server_key: "0d26722c67e88caca5bf4f0c90e8ee01"
server_key_iv: "abd205cbce8dd9fca8cd499027952d9b"
server_key_version: "encryptionkey@0"
I20260811 03:11:06.151412 12524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "2e59479f2b704c5f87ec22c5b0c2c489"
format_stamp: "Formatted at 2026-08-11 03:11:06 on dist-test-slave-tm1g"
server_key: "0d26722c67e88caca5bf4f0c90e8ee01"
server_key_iv: "abd205cbce8dd9fca8cd499027952d9b"
server_key_version: "encryptionkey@0"
I20260811 03:11:06.156905 12524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 03:11:06.160578 12542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:06.161985 12524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:11:06.162168 12524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "2e59479f2b704c5f87ec22c5b0c2c489"
format_stamp: "Formatted at 2026-08-11 03:11:06 on dist-test-slave-tm1g"
server_key: "0d26722c67e88caca5bf4f0c90e8ee01"
server_key_iv: "abd205cbce8dd9fca8cd499027952d9b"
server_key_version: "encryptionkey@0"
I20260811 03:11:06.162477 12524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:06.187868 12524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:06.192926 12524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:06.193252 12524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:06.194397 12524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:06.195930 12524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:06.196005 12524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:06.196137 12524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:06.196188 12524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:06.199410 12366 heartbeater.cc:503] Master 127.11.60.62:46723 was elected leader, sending a full tablet report...
I20260811 03:11:06.239389 12524 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:46881
I20260811 03:11:06.239445 12664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:46881 every 8 connection(s)
I20260811 03:11:06.240775 12524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:06.242637 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 12524
I20260811 03:11:06.242926 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:06.243510 11504 external_mini_cluster.cc:1428] Setting key 270c58064dc2a6868f956526bac2c42b
Aug 11 03:11:06 dist-test-slave-tm1g krb5kdc[12110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417866, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:06.266179 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.244389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:46567 (local address 127.11.60.62:46723)
0811 03:11:06.250831 (+ 6442us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.250836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.250852 (+ 16us) server_negotiation.cc:407] Connection header received
0811 03:11:06.250899 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.250902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.250964 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.251057 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:06.252746 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.253671 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.255364 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.255778 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.255875 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.260026 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:06.260032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:06.260035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:06.260068 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:06.262898 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.263674 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.263684 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.263686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.263810 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.264276 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.264281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.264283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.264555 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:06.264731 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.265511 (+ 780us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.265762 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6335,"thread_start_us":6252,"threads_started":1}
I20260811 03:11:06.267237 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.244942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:46723 (local address 127.11.60.3:46567)
0811 03:11:06.246518 (+ 1576us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:06.246655 (+ 137us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:06.250208 (+ 3553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:06.251319 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:06.251328 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:06.251939 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:06.252562 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.252580 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.253826 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.253830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.255236 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.255249 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.256003 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.256008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.256272 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.257028 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:06.257051 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:06.259780 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:06.263045 (+ 3265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:06.263054 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:06.263072 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:06.263539 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:06.263934 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:06.263939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:06.263943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:06.264147 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:06.264695 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:06.264704 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:06.265109 (+ 405us) client_negotiation.cc:769] Sending connection context
0811 03:11:06.265399 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 03:11:06.265977 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"thread_start_us":797,"threads_started":1}
I20260811 03:11:06.268781 12665 heartbeater.cc:347] Connected to a master server at 127.11.60.62:46723
I20260811 03:11:06.269131 12665 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:06.269938 12665 heartbeater.cc:511] Master 127.11.60.62:46723 requested a full tablet report, sending...
I20260811 03:11:06.271694 12160 ts_manager.cc:194] Registered new tserver with Master: 2e59479f2b704c5f87ec22c5b0c2c489 (127.11.60.3:46881)
I20260811 03:11:06.272711 12160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:46567
I20260811 03:11:06.272852 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:11:06.295140 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.280338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60942 (local address 127.11.60.62:46723)
0811 03:11:06.280603 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.280611 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.281336 (+ 725us) server_negotiation.cc:407] Connection header received
0811 03:11:06.281509 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.281513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.281580 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.281669 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:06.283335 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.284200 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.285631 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.286150 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.286250 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.288771 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:06.288776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:06.288780 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:06.288812 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:06.292117 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.293009 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.293014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.293016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.293168 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.293596 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.293600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.293602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.293938 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:06.294149 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.294603 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.294784 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 03:11:06.302858 12160 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:11:06.310130 12160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:06.337774 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.328510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52772 (local address 127.11.60.2:36287)
0811 03:11:06.329687 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.329695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.329721 (+ 26us) server_negotiation.cc:407] Connection header received
0811 03:11:06.329814 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.329820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.330034 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.331095 (+ 1061us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:06.331351 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.332855 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.335541 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.336542 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.336695 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.336928 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.337261 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.337448 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":843,"threads_started":1}
I20260811 03:11:06.338609 12682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.324435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:46881 (local address 127.0.0.1:37076)
0811 03:11:06.325249 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:06.325306 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:06.325495 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:06.330003 (+ 4508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:06.330008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:06.330044 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:06.330678 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.330694 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.332371 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.332379 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.335582 (+ 3203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.335596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.337662 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.337669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.337850 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.337886 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 03:11:06.337965 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 03:11:06.338106 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":102,"thread_start_us":419,"threads_started":1}
I20260811 03:11:06.337877 12686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.327983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:36287 (local address 127.0.0.1:52772)
0811 03:11:06.328923 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:06.328947 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:06.329121 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:06.330256 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:06.330262 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:06.330288 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:06.330904 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.330918 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.332990 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.333004 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.335379 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.335391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.336810 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.336818 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.337026 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.337063 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 03:11:06.337205 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 03:11:06.337404 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":5,"thread_start_us":722,"threads_started":1}
I20260811 03:11:06.339143 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.324746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.11.60.3:46881)
0811 03:11:06.328175 (+ 3429us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.328184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.328219 (+ 35us) server_negotiation.cc:407] Connection header received
0811 03:11:06.328324 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.328334 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.328639 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.330920 (+ 2281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:06.331197 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.332229 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.335758 (+ 3529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.337420 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.338224 (+ 804us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.338499 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.338584 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.338775 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3217,"thread_start_us":3027,"threads_started":1}
I20260811 03:11:06.340524 12683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.324578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:39873 (local address 127.0.0.1:56022)
0811 03:11:06.328506 (+ 3928us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:06.328537 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:06.328838 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:06.329854 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:06.329862 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:06.329891 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:06.330474 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.330491 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.332616 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.332620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.337176 (+ 4556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.337192 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.339685 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.339692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.339852 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.339881 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 03:11:06.339968 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 03:11:06.340101 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3783,"mutex_wait_us":100,"thread_start_us":3638,"threads_started":1}
I20260811 03:11:06.340770 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.325816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56022 (local address 127.11.60.1:39873)
0811 03:11:06.327584 (+ 1768us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.327591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.328721 (+ 1130us) server_negotiation.cc:407] Connection header received
0811 03:11:06.329105 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.329114 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.329359 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.329539 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:06.330720 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.332468 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.337392 (+ 4924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.339479 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.339624 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.339839 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.340072 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.340251 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1598,"thread_start_us":1481,"threads_started":1}
I20260811 03:11:06.343065 12594 tablet_service.cc:1467] Processing CreateTablet for tablet 9b16c5d20d7346dbb2c3fc18c76e209e (DEFAULT_TABLE table=test-table [id=c446c17191d742b7a49203109b38f2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:06.343079 12444 tablet_service.cc:1467] Processing CreateTablet for tablet 9b16c5d20d7346dbb2c3fc18c76e209e (DEFAULT_TABLE table=test-table [id=c446c17191d742b7a49203109b38f2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:06.344578 12594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b16c5d20d7346dbb2c3fc18c76e209e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:06.344594 12444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b16c5d20d7346dbb2c3fc18c76e209e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:06.344472 12299 tablet_service.cc:1467] Processing CreateTablet for tablet 9b16c5d20d7346dbb2c3fc18c76e209e (DEFAULT_TABLE table=test-table [id=c446c17191d742b7a49203109b38f2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:06.345856 12299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b16c5d20d7346dbb2c3fc18c76e209e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:06.359279 12688 tablet_bootstrap.cc:492] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489: Bootstrap starting.
I20260811 03:11:06.362479 12689 tablet_bootstrap.cc:492] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76: Bootstrap starting.
I20260811 03:11:06.364342 12690 tablet_bootstrap.cc:492] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c: Bootstrap starting.
I20260811 03:11:06.364846 12688 tablet_bootstrap.cc:654] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:06.366834 12689 tablet_bootstrap.cc:654] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:06.367493 12688 log.cc:824] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:06.368319 12689 log.cc:824] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:06.368469 12690 tablet_bootstrap.cc:654] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:06.369765 12690 log.cc:824] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:06.371287 12688 tablet_bootstrap.cc:492] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489: No bootstrap required, opened a new log
I20260811 03:11:06.371640 12688 ts_tablet_manager.cc:1397] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260811 03:11:06.373046 12689 tablet_bootstrap.cc:492] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76: No bootstrap required, opened a new log
I20260811 03:11:06.373355 12689 ts_tablet_manager.cc:1397] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 03:11:06.373354 12690 tablet_bootstrap.cc:492] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c: No bootstrap required, opened a new log
I20260811 03:11:06.373720 12690 ts_tablet_manager.cc:1397] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 03:11:06.382771 12689 raft_consensus.cc:348] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46881 } } peers { permanent_uuid: "449f757ebcc346ae9e4739b765a7598c" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39873 } } peers { permanent_uuid: "cea7f9be783046acb95bcdbad425ec76" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36287 } }
I20260811 03:11:06.383265 12689 raft_consensus.cc:374] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:06.383364 12689 raft_consensus.cc:729] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea7f9be783046acb95bcdbad425ec76, State: Initialized, Role: FOLLOWER
I20260811 03:11:06.384087 12689 consensus_queue.cc:260] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46881 } } peers { permanent_uuid: "449f757ebcc346ae9e4739b765a7598c" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39873 } } peers { permanent_uuid: "cea7f9be783046acb95bcdbad425ec76" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36287 } }
I20260811 03:11:06.384912 12688 raft_consensus.cc:348] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46881 } } peers { permanent_uuid: "449f757ebcc346ae9e4739b765a7598c" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39873 } } peers { permanent_uuid: "cea7f9be783046acb95bcdbad425ec76" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36287 } }
I20260811 03:11:06.385452 12688 raft_consensus.cc:374] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:06.385622 12688 raft_consensus.cc:729] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e59479f2b704c5f87ec22c5b0c2c489, State: Initialized, Role: FOLLOWER
I20260811 03:11:06.385842 12517 heartbeater.cc:503] Master 127.11.60.62:46723 was elected leader, sending a full tablet report...
I20260811 03:11:06.386240 12688 consensus_queue.cc:260] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46881 } } peers { permanent_uuid: "449f757ebcc346ae9e4739b765a7598c" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39873 } } peers { permanent_uuid: "cea7f9be783046acb95bcdbad425ec76" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36287 } }
I20260811 03:11:06.386957 12689 ts_tablet_manager.cc:1428] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260811 03:11:06.386582 12690 raft_consensus.cc:348] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46881 } } peers { permanent_uuid: "449f757ebcc346ae9e4739b765a7598c" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39873 } } peers { permanent_uuid: "cea7f9be783046acb95bcdbad425ec76" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36287 } }
I20260811 03:11:06.387131 12690 raft_consensus.cc:374] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:06.387248 12690 raft_consensus.cc:729] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 449f757ebcc346ae9e4739b765a7598c, State: Initialized, Role: FOLLOWER
I20260811 03:11:06.388017 12690 consensus_queue.cc:260] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46881 } } peers { permanent_uuid: "449f757ebcc346ae9e4739b765a7598c" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39873 } } peers { permanent_uuid: "cea7f9be783046acb95bcdbad425ec76" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36287 } }
I20260811 03:11:06.390416 12690 ts_tablet_manager.cc:1428] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c: Time spent starting tablet: real 0.017s user 0.018s sys 0.001s
I20260811 03:11:06.391711 12665 heartbeater.cc:503] Master 127.11.60.62:46723 was elected leader, sending a full tablet report...
I20260811 03:11:06.392477 12688 ts_tablet_manager.cc:1428] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
W20260811 03:11:06.417235 12368 tablet.cc:2369] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:11:06.451316 12518 tablet.cc:2369] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:11:06.492708 12666 tablet.cc:2369] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:06.698330 12695 raft_consensus.cc:482] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:06.698747 12695 raft_consensus.cc:504] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46881 } } peers { permanent_uuid: "449f757ebcc346ae9e4739b765a7598c" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39873 } } peers { permanent_uuid: "cea7f9be783046acb95bcdbad425ec76" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36287 } }
I20260811 03:11:06.700875 12695 leader_election.cc:288] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 449f757ebcc346ae9e4739b765a7598c (127.11.60.1:39873), cea7f9be783046acb95bcdbad425ec76 (127.11.60.2:36287)
I20260811 03:11:06.707877 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.701715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:56715 (local address 127.11.60.1:39873)
0811 03:11:06.701986 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.701992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.702012 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:11:06.702090 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.702097 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.702225 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.702355 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:06.702789 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.703924 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.705943 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.706862 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.707019 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.707124 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.707489 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.707640 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 03:11:06.708338 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.701110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:39873 (local address 127.11.60.3:56715)
0811 03:11:06.701467 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:06.701489 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:06.701624 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:06.702380 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:06.702386 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:06.702418 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:06.702620 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.702630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.704084 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.704089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.705776 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.705792 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.707032 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.707037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.707296 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.707336 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 03:11:06.707442 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 03:11:06.707586 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260811 03:11:06.709549 12320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b16c5d20d7346dbb2c3fc18c76e209e" candidate_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "449f757ebcc346ae9e4739b765a7598c" is_pre_election: true
I20260811 03:11:06.709924 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.701900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:43637 (local address 127.11.60.2:36287)
0811 03:11:06.702096 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.702102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.702258 (+ 156us) server_negotiation.cc:407] Connection header received
0811 03:11:06.702586 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.702591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.702657 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.702715 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:06.703457 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.704539 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.707350 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.708879 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.709043 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.709194 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.709528 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.709659 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 03:11:06.710155 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.701306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:36287 (local address 127.11.60.3:43637)
0811 03:11:06.702082 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:06.702098 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:06.702398 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:06.702896 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:06.702898 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:06.702932 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:06.703277 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.703285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.704727 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.704732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.707182 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:06.707200 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.709191 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.709198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:06.709360 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.709383 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 03:11:06.709490 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 03:11:06.709623 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":33,"thread_start_us":765,"threads_started":1}
I20260811 03:11:06.710563 12320 raft_consensus.cc:2454] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e59479f2b704c5f87ec22c5b0c2c489 in term 0.
I20260811 03:11:06.711382 12466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b16c5d20d7346dbb2c3fc18c76e209e" candidate_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea7f9be783046acb95bcdbad425ec76" is_pre_election: true
I20260811 03:11:06.711668 12544 leader_election.cc:302] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e59479f2b704c5f87ec22c5b0c2c489, 449f757ebcc346ae9e4739b765a7598c; no voters:
I20260811 03:11:06.711953 12466 raft_consensus.cc:2454] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e59479f2b704c5f87ec22c5b0c2c489 in term 0.
I20260811 03:11:06.712111 12695 raft_consensus.cc:2781] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:06.712292 12695 raft_consensus.cc:482] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:06.712436 12695 raft_consensus.cc:3037] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:06.715059 12695 raft_consensus.cc:504] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46881 } } peers { permanent_uuid: "449f757ebcc346ae9e4739b765a7598c" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39873 } } peers { permanent_uuid: "cea7f9be783046acb95bcdbad425ec76" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36287 } }
I20260811 03:11:06.715657 12695 leader_election.cc:288] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [CANDIDATE]: Term 1 election: Requested vote from peers 449f757ebcc346ae9e4739b765a7598c (127.11.60.1:39873), cea7f9be783046acb95bcdbad425ec76 (127.11.60.2:36287)
I20260811 03:11:06.716137 12320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b16c5d20d7346dbb2c3fc18c76e209e" candidate_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "449f757ebcc346ae9e4739b765a7598c"
I20260811 03:11:06.716200 12466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b16c5d20d7346dbb2c3fc18c76e209e" candidate_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea7f9be783046acb95bcdbad425ec76"
I20260811 03:11:06.716382 12466 raft_consensus.cc:3037] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:06.716383 12320 raft_consensus.cc:3037] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:06.720173 12466 raft_consensus.cc:2454] T 9b16c5d20d7346dbb2c3fc18c76e209e P cea7f9be783046acb95bcdbad425ec76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e59479f2b704c5f87ec22c5b0c2c489 in term 1.
I20260811 03:11:06.720280 12320 raft_consensus.cc:2454] T 9b16c5d20d7346dbb2c3fc18c76e209e P 449f757ebcc346ae9e4739b765a7598c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e59479f2b704c5f87ec22c5b0c2c489 in term 1.
I20260811 03:11:06.720953 12545 leader_election.cc:302] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e59479f2b704c5f87ec22c5b0c2c489, cea7f9be783046acb95bcdbad425ec76; no voters:
I20260811 03:11:06.721319 12695 raft_consensus.cc:2781] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:06.721993 12695 raft_consensus.cc:686] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [term 1 LEADER]: Becoming Leader. State: Replica: 2e59479f2b704c5f87ec22c5b0c2c489, State: Running, Role: LEADER
I20260811 03:11:06.722586 12695 consensus_queue.cc:237] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46881 } } peers { permanent_uuid: "449f757ebcc346ae9e4739b765a7598c" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39873 } } peers { permanent_uuid: "cea7f9be783046acb95bcdbad425ec76" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36287 } }
I20260811 03:11:06.728119 12159 catalog_manager.cc:5495] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e59479f2b704c5f87ec22c5b0c2c489 (127.11.60.3). New cstate: current_term: 1 leader_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e59479f2b704c5f87ec22c5b0c2c489" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "449f757ebcc346ae9e4739b765a7598c" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea7f9be783046acb95bcdbad425ec76" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36287 } health_report { overall_health: UNKNOWN } } }
Aug 11 03:11:06 dist-test-slave-tm1g krb5kdc[12110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:06.806553 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.787266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56028 (local address 127.11.60.1:39873)
0811 03:11:06.787551 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.787567 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.789027 (+ 1460us) server_negotiation.cc:407] Connection header received
0811 03:11:06.789181 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.789186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.789272 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.789363 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:06.791810 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.792667 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.794121 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.794780 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.794897 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.800111 (+ 5214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:06.800130 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:06.800147 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:06.800190 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:06.803560 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.804381 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.804390 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.804392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.804555 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.804923 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.804927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.804929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.805348 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:06.805596 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.805897 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.806072 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 03:11:06 dist-test-slave-tm1g krb5kdc[12110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 03:11:06 dist-test-slave-tm1g krb5kdc[12110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:06.867511 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.852023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.11.60.1:39873)
0811 03:11:06.852312 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.852321 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.852603 (+ 282us) server_negotiation.cc:407] Connection header received
0811 03:11:06.852846 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.852851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.852918 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.853028 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:06.854551 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.855314 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.856757 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.857212 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.857356 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.861627 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:06.861633 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:06.861636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:06.861675 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:06.864656 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.865360 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.865367 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.865371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.865539 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.866004 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.866008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.866010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.866411 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:06.866679 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.866927 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.867195 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 03:11:06 dist-test-slave-tm1g krb5kdc[12110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 03:11:06 dist-test-slave-tm1g krb5kdc[12110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:06.924208 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.908843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.11.60.1:39873)
0811 03:11:06.909113 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.909121 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.909456 (+ 335us) server_negotiation.cc:407] Connection header received
0811 03:11:06.909652 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.909656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.909722 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.909834 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:06.911342 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.912089 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.913411 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.913962 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.914116 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.918685 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:06.918690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:06.918693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:06.918727 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:06.921362 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.922041 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.922048 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.922051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.922194 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.922601 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.922608 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.922611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.922885 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:06.923242 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.923566 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.923827 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260811 03:11:06.926476 12277 tablet_service.cc:2908] Rejecting scan request for tablet 9b16c5d20d7346dbb2c3fc18c76e209e: Uninitialized: clean time has not yet been initialized
I20260811 03:11:06.930037 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 12226
I20260811 03:11:06.944391 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:06.931009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56068 (local address 127.11.60.1:39873)
0811 03:11:06.931238 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:06.931247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:06.931595 (+ 348us) server_negotiation.cc:407] Connection header received
0811 03:11:06.931791 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:06.931795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:06.931873 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:06.931962 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:06.933528 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.934241 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.935527 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:06.935971 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:06.936088 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:06.938944 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:06.938950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:06.938953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:06.938984 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:06.941472 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.942267 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.942272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.942274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.942665 (+ 391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:06.943081 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:06.943085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:06.943087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:06.943348 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:06.943585 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:06.943871 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 03:11:06.944046 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 03:11:06.945035 12361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:07.037460 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 12226
I20260811 03:11:07.057518 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 12375
Aug 11 03:11:07 dist-test-slave-tm1g krb5kdc[12110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:11:07.074234 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:07.058476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52778 (local address 127.11.60.2:36287)
0811 03:11:07.058660 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:07.058665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:07.058684 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:11:07.058801 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:07.058806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:07.058887 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:07.058991 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:07.060343 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:07.061125 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:07.062589 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:07.063003 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:07.063132 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:07.067895 (+ 4763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:07.067912 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:07.067929 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:07.067968 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:07.071128 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:07.072045 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:07.072053 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:07.072055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:07.072195 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:07.072659 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:07.072663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:07.072665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:07.073063 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:07.073295 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:07.073671 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 03:11:07.073851 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 03:11:07.075058 12511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:07.166642 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 12375
W20260811 03:11:07.186642 12545 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.60.2:36287 (error 108)
I20260811 03:11:07.186954 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 12524
I20260811 03:11:07.187008 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:07.186080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:39873 (local address 127.11.60.3:60963)
0811 03:11:07.186527 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:07.186804 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:39873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 03:11:07.188841 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:07.188343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:36287 (local address 127.11.60.3:46779)
0811 03:11:07.188562 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:07.188649 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.2:36287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 03:11:07.189067 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:07.188635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:39873 (local address 127.11.60.3:39455)
0811 03:11:07.188790 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:07.188897 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:39873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 03:11:07.189355 12544 consensus_peers.cc:487] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 -> Peer 449f757ebcc346ae9e4739b765a7598c (127.11.60.1:39873): Couldn't send request to peer 449f757ebcc346ae9e4739b765a7598c. Status: Network error: Client connection negotiation failed: client connection to 127.11.60.1:39873: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 03:11:07.189366 12545 consensus_peers.cc:487] T 9b16c5d20d7346dbb2c3fc18c76e209e P 2e59479f2b704c5f87ec22c5b0c2c489 -> Peer cea7f9be783046acb95bcdbad425ec76 (127.11.60.2:36287): Couldn't send request to peer cea7f9be783046acb95bcdbad425ec76. Status: Network error: Client connection negotiation failed: client connection to 127.11.60.2:36287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 11 03:11:07 dist-test-slave-tm1g krb5kdc[12110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:11:07.203368 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:07.187923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37080 (local address 127.11.60.3:46881)
0811 03:11:07.188558 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:07.188566 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:07.188591 (+ 25us) server_negotiation.cc:407] Connection header received
0811 03:11:07.188651 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:07.188654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:07.188790 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:07.188911 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:07.190489 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:07.191300 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:07.192649 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:07.193079 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:07.193176 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:07.197386 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:07.197399 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:07.197442 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:07.197477 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:07.200468 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:07.201160 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:07.201167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:07.201170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:07.201381 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:07.201799 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:07.201803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:07.201805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:07.202148 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:07.202539 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:07.202738 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 03:11:07.202949 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":545,"threads_started":1}
I20260811 03:11:07.204185 12658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:07.297022 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 12524
I20260811 03:11:07.317046 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 12124
I20260811 03:11:07.318449 12192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:07.429388 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 12124
2026-08-11T03:11:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4710 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:11:07 dist-test-slave-tm1g krb5kdc[12722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:11:07 dist-test-slave-tm1g krb5kdc[12722](info): set up 2 sockets
Aug 11 03:11:07 dist-test-slave-tm1g krb5kdc[12722](info): commencing operation
krb5kdc: starting...
W20260811 03:11:08.589192 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:11:08 dist-test-slave-tm1g krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:11:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:11:08.806257 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42599
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:42599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:09.142668 12736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:09.143036 12736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:09.143196 12736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:09.150540 12736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:09.150694 12736 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 03:11:09.150797 12736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:09.150923 12736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:09.151010 12736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:09.151103 12736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:09.162611 12736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:42599
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:09.164052 12736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:09.165464 12736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:09.178125 12747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:09.178171 12744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:09.178223 12743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:09.184173 12736 server_base.cc:1034] running on GCE node
I20260811 03:11:09.185161 12736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:09.186877 12736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:09.188143 12736 hybrid_clock.cc:648] HybridClock initialized: now 1786417869188062 us; error 60 us; skew 500 ppm
I20260811 03:11:09.188510 12736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:09 dist-test-slave-tm1g krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417869, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:09.191958 12736 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:11:09.193558 12736 webserver.cc:460] Webserver started at http://127.11.60.62:34059/ using document root <none> and password file <none>
I20260811 03:11:09.194406 12736 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:09.194543 12736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:09.194893 12736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:09.197996 12736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05"
format_stamp: "Formatted at 2026-08-11 03:11:09 on dist-test-slave-tm1g"
server_key: "541fc30f1eabc71168bd60161137ca2d"
server_key_iv: "fb258260a0d766dd3417353d6dc7eafc"
server_key_version: "encryptionkey@0"
I20260811 03:11:09.198908 12736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05"
format_stamp: "Formatted at 2026-08-11 03:11:09 on dist-test-slave-tm1g"
server_key: "541fc30f1eabc71168bd60161137ca2d"
server_key_iv: "fb258260a0d766dd3417353d6dc7eafc"
server_key_version: "encryptionkey@0"
I20260811 03:11:09.204416 12736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:11:09.208070 12756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:09.209393 12736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:11:09.209583 12736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05"
format_stamp: "Formatted at 2026-08-11 03:11:09 on dist-test-slave-tm1g"
server_key: "541fc30f1eabc71168bd60161137ca2d"
server_key_iv: "fb258260a0d766dd3417353d6dc7eafc"
server_key_version: "encryptionkey@0"
I20260811 03:11:09.209810 12736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:09.229274 12736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:09.233845 12736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:09.234170 12736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:09.256554 12736 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:42599
I20260811 03:11:09.256580 12823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:42599 every 8 connection(s)
I20260811 03:11:09.258080 12736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:09.261901 12824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:09.268237 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 12736
I20260811 03:11:09.268518 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:09.269158 11504 external_mini_cluster.cc:1428] Setting key 7e35e9253481ed3b42974a3c3b1de007
I20260811 03:11:09.272675 12824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05: Bootstrap starting.
I20260811 03:11:09.276489 12824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:09.277642 12824 log.cc:824] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:09.280553 12824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05: No bootstrap required, opened a new log
Aug 11 03:11:09 dist-test-slave-tm1g krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:09.288179 12824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42599 } }
I20260811 03:11:09.288640 12824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:09.288772 12824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44bf6f1690e14f3e91ccbe14ac7c8e05, State: Initialized, Role: FOLLOWER
I20260811 03:11:09.289444 12824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42599 } }
I20260811 03:11:09.289686 12824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:09.289834 12824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:09.290027 12824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:09.292164 12828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:09.271182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58490 (local address 127.11.60.62:42599)
0811 03:11:09.272045 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:09.272057 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:09.272099 (+ 42us) server_negotiation.cc:407] Connection header received
0811 03:11:09.273924 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:09.273946 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:09.274406 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:09.274741 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:09.276714 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:09.278123 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:09.279434 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:09.280092 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:09.280291 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:09.284437 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:09.284448 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:09.284462 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:09.284508 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:09.287764 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:09.288498 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:09.288504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:09.288510 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:09.288766 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:09.289237 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:09.289242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:09.289245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:09.289918 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:09.290242 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:09.290698 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 03:11:09.291147 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":442,"threads_started":1}
I20260811 03:11:09.292681 12824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42599 } }
I20260811 03:11:09.293259 12824 leader_election.cc:302] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44bf6f1690e14f3e91ccbe14ac7c8e05; no voters:
I20260811 03:11:09.293949 12824 leader_election.cc:288] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:09.294067 12831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:09.296128 12824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:09.296469 12831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [term 1 LEADER]: Becoming Leader. State: Replica: 44bf6f1690e14f3e91ccbe14ac7c8e05, State: Running, Role: LEADER
I20260811 03:11:09.296996 12831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42599 } }
I20260811 03:11:09.300696 12833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44bf6f1690e14f3e91ccbe14ac7c8e05. Latest consensus state: current_term: 1 leader_uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42599 } } }
I20260811 03:11:09.301116 12833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:09.300685 12832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bf6f1690e14f3e91ccbe14ac7c8e05" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42599 } } }
I20260811 03:11:09.301751 12832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:09.304418 12839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:09.309393 12839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:09.319145 12839 catalog_manager.cc:1347] Generated new cluster ID: c13b82b4c9f94ccbb46f0ae0d965e633
I20260811 03:11:09.319268 12839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:09.350545 12839 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:09.351972 12839 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:09.361231 12839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05: Generated new TSK 0
I20260811 03:11:09.362123 12839 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:11:09.418116 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42599
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:09.756932 12854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:09.757210 12854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:09.757308 12854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:09.764590 12854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:09.764709 12854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:09.764794 12854 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 03:11:09.764873 12854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:09.775341 12854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.60.62:42599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:09.776682 12854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:09.777976 12854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:09.791265 12861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:09.791193 12863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:09.791229 12866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:09.792632 12854 server_base.cc:1034] running on GCE node
I20260811 03:11:09.793336 12854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:09.794319 12854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:09.795557 12854 hybrid_clock.cc:648] HybridClock initialized: now 1786417869795475 us; error 54 us; skew 500 ppm
I20260811 03:11:09.795857 12854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:09 dist-test-slave-tm1g krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417869, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:09.804728 12854 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:11:09.806594 12854 webserver.cc:460] Webserver started at http://127.11.60.1:46001/ using document root <none> and password file <none>
I20260811 03:11:09.807375 12854 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:09.807509 12854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:09.807842 12854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:09.810978 12854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "209b6611c83a494b8cdbaef7d0d7ef51"
format_stamp: "Formatted at 2026-08-11 03:11:09 on dist-test-slave-tm1g"
server_key: "e0b264f52ae573a9b794455049a48c14"
server_key_iv: "7a4f8fc5bdc022d7f2e79d67fe5544e2"
server_key_version: "encryptionkey@0"
I20260811 03:11:09.811806 12854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "209b6611c83a494b8cdbaef7d0d7ef51"
format_stamp: "Formatted at 2026-08-11 03:11:09 on dist-test-slave-tm1g"
server_key: "e0b264f52ae573a9b794455049a48c14"
server_key_iv: "7a4f8fc5bdc022d7f2e79d67fe5544e2"
server_key_version: "encryptionkey@0"
I20260811 03:11:09.817399 12854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 03:11:09.820971 12873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:09.822273 12854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 03:11:09.822499 12854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "209b6611c83a494b8cdbaef7d0d7ef51"
format_stamp: "Formatted at 2026-08-11 03:11:09 on dist-test-slave-tm1g"
server_key: "e0b264f52ae573a9b794455049a48c14"
server_key_iv: "7a4f8fc5bdc022d7f2e79d67fe5544e2"
server_key_version: "encryptionkey@0"
I20260811 03:11:09.822726 12854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:09.846375 12854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:09.850513 12854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:09.850795 12854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:09.853499 12854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:09.853641 12854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:09.853791 12854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:09.853881 12854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 03:11:09 dist-test-slave-tm1g krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417869, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:09.878931 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:09.854485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:38617 (local address 127.11.60.62:42599)
0811 03:11:09.856680 (+ 2195us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:09.856686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:09.856709 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:11:09.858560 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:09.858566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:09.858669 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:09.858794 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:09.861266 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:09.862413 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:09.865655 (+ 3242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:09.866105 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:09.866253 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:09.871607 (+ 5354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:09.871614 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:09.871630 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:09.871679 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:09.874641 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:09.875900 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:09.875907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:09.875912 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:09.876066 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:09.876611 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:09.876616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:09.876619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:09.877005 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:09.877161 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:09.878030 (+ 869us) server_negotiation.cc:299] Negotiation successful
0811 03:11:09.878401 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2059,"thread_start_us":1947,"threads_started":1}
I20260811 03:11:09.879776 12884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:09.854982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42599 (local address 127.11.60.1:38617)
0811 03:11:09.855864 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:09.855909 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:09.858171 (+ 2262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:09.859095 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:09.859109 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:09.859958 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:09.861068 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:09.861100 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:09.862629 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:09.862638 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:09.865474 (+ 2836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:09.865497 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:09.866448 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:09.866460 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:09.866847 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:09.867850 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:09.867891 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:09.871243 (+ 3352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:09.874852 (+ 3609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:09.874863 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:09.874885 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:09.875753 (+ 868us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:09.876195 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:09.876200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:09.876204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:09.876448 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:09.877239 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:09.877245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:09.877604 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 03:11:09.877845 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 03:11:09.878360 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":425,"threads_started":1}
I20260811 03:11:09.900906 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:09.885882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:36471 (local address 127.11.60.62:42599)
0811 03:11:09.886113 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:09.886122 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:09.886917 (+ 795us) server_negotiation.cc:407] Connection header received
0811 03:11:09.887108 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:09.887114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:09.887197 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:09.887301 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:09.888248 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:09.889036 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:09.890355 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:09.891312 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:09.891643 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:09.894271 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:09.894276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:09.894279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:09.894353 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:09.897783 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:09.898892 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:09.898899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:09.898905 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:09.899050 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:09.899522 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:09.899525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:09.899527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:09.899792 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:09.899907 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:09.900496 (+ 589us) server_negotiation.cc:299] Negotiation successful
0811 03:11:09.900664 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:11:09.901556 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:09.885918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42599 (local address 127.11.60.1:36471)
0811 03:11:09.886808 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:09.886826 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:09.886985 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:09.887327 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:09.887331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:09.887808 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:09.888104 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:09.888113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:09.889169 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:09.889175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:09.890194 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:09.890206 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:09.891694 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:09.891701 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:09.892027 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:09.892834 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:09.892854 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:09.894056 (+ 1202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:09.897929 (+ 3873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:09.897935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:09.897939 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:09.898747 (+ 808us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:09.899175 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:09.899183 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:09.899189 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:09.899408 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:09.899913 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:09.899921 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:09.900204 (+ 283us) client_negotiation.cc:769] Sending connection context
0811 03:11:09.900402 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 03:11:09.900843 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"thread_start_us":610,"threads_started":1}
I20260811 03:11:09.916597 12854 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:41505
I20260811 03:11:09.916626 13009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:41505 every 8 connection(s)
I20260811 03:11:09.917912 12854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:09.919377 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 12854
I20260811 03:11:09.919616 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:09.920405 13010 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42599
I20260811 03:11:09.920469 11504 external_mini_cluster.cc:1428] Setting key ca984edf00cf59839dbe6f7a638ea63e
I20260811 03:11:09.920647 13010 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:09.921339 13010 heartbeater.cc:511] Master 127.11.60.62:42599 requested a full tablet report, sending...
I20260811 03:11:09.923309 12774 ts_manager.cc:194] Registered new tserver with Master: 209b6611c83a494b8cdbaef7d0d7ef51 (127.11.60.1:41505)
I20260811 03:11:09.924767 12774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:38617
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:11:09.966048 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42599
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:10.300639 13014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:10.300928 13014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:10.301033 13014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:10.308462 13014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:10.308569 13014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:10.308651 13014 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 03:11:10.308732 13014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:10.318934 13014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.60.62:42599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:10.320266 13014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:10.321678 13014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:10.335223 13020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:10.335167 13022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:10.335198 13024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:10.336107 13014 server_base.cc:1034] running on GCE node
I20260811 03:11:10.336803 13014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:10.337667 13014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:10.338881 13014 hybrid_clock.cc:648] HybridClock initialized: now 1786417870338830 us; error 32 us; skew 500 ppm
I20260811 03:11:10.339155 13014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:10 dist-test-slave-tm1g krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:10.342379 13014 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:11:10.343896 13014 webserver.cc:460] Webserver started at http://127.11.60.2:43587/ using document root <none> and password file <none>
I20260811 03:11:10.344655 13014 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:10.344743 13014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:10.345090 13014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:10.348115 13014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "b14679b0b266451b8fa59baea0068ccd"
format_stamp: "Formatted at 2026-08-11 03:11:10 on dist-test-slave-tm1g"
server_key: "6524ed09991bcb3e19e5b0bf3303febd"
server_key_iv: "f56889e7a2ce9637b517f55d798de244"
server_key_version: "encryptionkey@0"
I20260811 03:11:10.348940 13014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "b14679b0b266451b8fa59baea0068ccd"
format_stamp: "Formatted at 2026-08-11 03:11:10 on dist-test-slave-tm1g"
server_key: "6524ed09991bcb3e19e5b0bf3303febd"
server_key_iv: "f56889e7a2ce9637b517f55d798de244"
server_key_version: "encryptionkey@0"
I20260811 03:11:10.354040 13014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 03:11:10.357664 13032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:10.359112 13014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:10.359295 13014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "b14679b0b266451b8fa59baea0068ccd"
format_stamp: "Formatted at 2026-08-11 03:11:10 on dist-test-slave-tm1g"
server_key: "6524ed09991bcb3e19e5b0bf3303febd"
server_key_iv: "f56889e7a2ce9637b517f55d798de244"
server_key_version: "encryptionkey@0"
I20260811 03:11:10.359539 13014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:10.381866 13014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:10.386731 13014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:10.387033 13014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:10.389784 13014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:10.389923 13014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:10.390077 13014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:10.390151 13014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 03:11:10 dist-test-slave-tm1g krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:10.413784 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:10.390710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:45005 (local address 127.11.60.62:42599)
0811 03:11:10.390917 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:10.390923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:10.392368 (+ 1445us) server_negotiation.cc:407] Connection header received
0811 03:11:10.394481 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:10.394517 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:10.394622 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:10.394736 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:10.397070 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.398216 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.401585 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.402219 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.402382 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:10.407282 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:10.407288 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:10.407299 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:10.407334 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:10.410013 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:10.411000 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:10.411007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:10.411011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:10.411144 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:10.411626 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:10.411633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:10.411636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:10.411937 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:10.412110 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:10.413147 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0811 03:11:10.413419 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 03:11:10.415400 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:10.391362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42599 (local address 127.11.60.2:45005)
0811 03:11:10.392204 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:10.392251 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:10.394155 (+ 1904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:10.395004 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:10.395013 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:10.395838 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:10.396864 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:10.396896 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.398440 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.398449 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:10.401385 (+ 2936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:10.401411 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.402542 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.402551 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:10.402900 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:10.403808 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:10.403858 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:10.407062 (+ 3204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:10.410164 (+ 3102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:10.410174 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:10.410190 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:10.410804 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:10.411268 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:10.411275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:10.411283 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:10.411490 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:10.412073 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:10.412082 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:10.412640 (+ 558us) client_negotiation.cc:769] Sending connection context
0811 03:11:10.413025 (+ 385us) client_negotiation.cc:240] Negotiation successful
0811 03:11:10.413769 (+ 744us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":393,"threads_started":1}
I20260811 03:11:10.439211 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:10.421954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:45121 (local address 127.11.60.62:42599)
0811 03:11:10.422146 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:10.422153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:10.422946 (+ 793us) server_negotiation.cc:407] Connection header received
0811 03:11:10.423171 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:10.423176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:10.423269 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:10.423379 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:10.424917 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.426150 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.428802 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.429298 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.429443 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:10.432076 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:10.432084 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:10.432090 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:10.432131 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:10.435708 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:10.436810 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:10.436818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:10.436825 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:10.437004 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:10.437549 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:10.437557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:10.437560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:10.437921 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:10.438038 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:10.438652 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 03:11:10.438931 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 03:11:10.439620 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:10.421997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42599 (local address 127.11.60.2:45121)
0811 03:11:10.422835 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:10.422851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:10.423025 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:10.423437 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:10.423450 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:10.424254 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:10.424726 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:10.424743 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.426340 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.426347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:10.428619 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:10.428639 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.429537 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.429547 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:10.429819 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:10.430713 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:10.430731 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:10.431765 (+ 1034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:10.435921 (+ 4156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:10.435929 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:10.435935 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:10.436585 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:10.437181 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:10.437187 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:10.437190 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:10.437397 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:10.438057 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:10.438064 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:10.438285 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 03:11:10.438485 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 03:11:10.438960 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":507,"threads_started":1}
I20260811 03:11:10.449636 13014 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:40795
I20260811 03:11:10.449666 13163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:40795 every 8 connection(s)
I20260811 03:11:10.450981 13014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:10.453203 13165 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42599
I20260811 03:11:10.453439 13165 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:10.454082 13165 heartbeater.cc:511] Master 127.11.60.62:42599 requested a full tablet report, sending...
I20260811 03:11:10.455511 12774 ts_manager.cc:194] Registered new tserver with Master: b14679b0b266451b8fa59baea0068ccd (127.11.60.2:40795)
I20260811 03:11:10.456382 12774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:45005
I20260811 03:11:10.456830 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 13014
I20260811 03:11:10.457043 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:10.457650 11504 external_mini_cluster.cc:1428] Setting key 4f0ec723b331e11433cf9a951929d497
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:11:10.505641 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42599
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:10.844858 13169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:10.845185 13169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:10.845386 13169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:10.852974 13169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:10.853302 13169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:10.853492 13169 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 03:11:10.853642 13169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:10.863837 13169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.60.62:42599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:10.865358 13169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:10.866808 13169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:10.880455 13175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:10.880499 13178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:10.880537 13176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:10.881403 13169 server_base.cc:1034] running on GCE node
I20260811 03:11:10.882110 13169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:10.883244 13169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:10.884471 13169 hybrid_clock.cc:648] HybridClock initialized: now 1786417870884387 us; error 60 us; skew 500 ppm
I20260811 03:11:10.884804 13169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:10 dist-test-slave-tm1g krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:10.888177 13169 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:11:10.889801 13169 webserver.cc:460] Webserver started at http://127.11.60.3:38589/ using document root <none> and password file <none>
I20260811 03:11:10.890666 13169 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:10.890762 13169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:10.891093 13169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:10.894133 13169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "4f33e4c458c949049f4a89d26cf15086"
format_stamp: "Formatted at 2026-08-11 03:11:10 on dist-test-slave-tm1g"
server_key: "d4e17f71f4a5aae9cc7fef139d5cb074"
server_key_iv: "d7a00dc677aaa1a6ae83a2de4cb1c1f3"
server_key_version: "encryptionkey@0"
I20260811 03:11:10.895064 13169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "4f33e4c458c949049f4a89d26cf15086"
format_stamp: "Formatted at 2026-08-11 03:11:10 on dist-test-slave-tm1g"
server_key: "d4e17f71f4a5aae9cc7fef139d5cb074"
server_key_iv: "d7a00dc677aaa1a6ae83a2de4cb1c1f3"
server_key_version: "encryptionkey@0"
I20260811 03:11:10.900395 13169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:11:10.904067 13185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:10.905480 13169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:11:10.905673 13169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "4f33e4c458c949049f4a89d26cf15086"
format_stamp: "Formatted at 2026-08-11 03:11:10 on dist-test-slave-tm1g"
server_key: "d4e17f71f4a5aae9cc7fef139d5cb074"
server_key_iv: "d7a00dc677aaa1a6ae83a2de4cb1c1f3"
server_key_version: "encryptionkey@0"
I20260811 03:11:10.905936 13169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:10.923870 13169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:10.927862 13010 heartbeater.cc:503] Master 127.11.60.62:42599 was elected leader, sending a full tablet report...
I20260811 03:11:10.928864 13169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:10.929147 13169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:10.932224 13169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:10.932354 13169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:10.932540 13169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:10.932605 13169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 03:11:10 dist-test-slave-tm1g krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:10.957355 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:10.932659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:54881 (local address 127.11.60.62:42599)
0811 03:11:10.932868 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:10.932875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:10.934348 (+ 1473us) server_negotiation.cc:407] Connection header received
0811 03:11:10.936727 (+ 2379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:10.936733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:10.936837 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:10.936951 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:10.939493 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.940859 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.943419 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.944229 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.944392 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:10.949285 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:10.949294 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:10.949301 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:10.949356 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:10.953261 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:10.954371 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:10.954376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:10.954378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:10.954517 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:10.955160 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:10.955166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:10.955171 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:10.955696 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:10.955909 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:10.956724 (+ 815us) server_negotiation.cc:299] Negotiation successful
0811 03:11:10.956972 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:11:10.958528 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:10.933354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42599 (local address 127.11.60.3:54881)
0811 03:11:10.934181 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:10.934221 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:10.936419 (+ 2198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:10.937224 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:10.937238 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:10.938228 (+ 990us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:10.939220 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:10.939257 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.941080 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.941086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:10.943259 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:10.943272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.944510 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.944521 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:10.944775 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:10.945596 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:10.945619 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:10.948972 (+ 3353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:10.953438 (+ 4466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:10.953451 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:10.953474 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:10.954125 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:10.954705 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:10.954711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:10.954717 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:10.955017 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:10.955874 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:10.955884 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:10.956251 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 03:11:10.956606 (+ 355us) client_negotiation.cc:240] Negotiation successful
0811 03:11:10.957173 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":392,"threads_started":1}
I20260811 03:11:10.983458 13169 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:43895
I20260811 03:11:10.983662 13316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:43895 every 8 connection(s)
I20260811 03:11:10.985183 13169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:10.987294 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 13169
I20260811 03:11:10.987520 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:10.987669 13317 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42599
I20260811 03:11:10.987918 13317 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:10.987951 11504 external_mini_cluster.cc:1428] Setting key fecb555bde8f80c3e655c539b7769a5e
I20260811 03:11:10.988809 13317 heartbeater.cc:511] Master 127.11.60.62:42599 requested a full tablet report, sending...
I20260811 03:11:10.990492 12773 ts_manager.cc:194] Registered new tserver with Master: 4f33e4c458c949049f4a89d26cf15086 (127.11.60.3:43895)
I20260811 03:11:10.990559 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:10.974215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42599 (local address 127.11.60.3:55447)
0811 03:11:10.975054 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:10.975073 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:10.975211 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:10.975676 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:10.975679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:10.976209 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:10.976552 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:10.976564 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.978334 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.978342 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:10.980015 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:10.980030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.981110 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.981118 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:10.981326 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:10.982069 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:10.982088 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:10.983397 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:10.987443 (+ 4046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:10.987449 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:10.987453 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:10.988042 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:10.988519 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:10.988524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:10.988527 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:10.988684 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:10.989377 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:10.989383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:10.989610 (+ 227us) client_negotiation.cc:769] Sending connection context
0811 03:11:10.989760 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 03:11:10.990107 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":469,"threads_started":1}
I20260811 03:11:10.991147 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:10.974201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:55447 (local address 127.11.60.62:42599)
0811 03:11:10.974491 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:10.974501 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:10.975171 (+ 670us) server_negotiation.cc:407] Connection header received
0811 03:11:10.975343 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:10.975353 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:10.975464 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:10.975612 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:10.976756 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.978066 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.980173 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:10.980863 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:10.981020 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:10.983645 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:10.983653 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:10.983658 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:10.983698 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:10.987286 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:10.988212 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:10.988217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:10.988220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:10.988376 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:10.988842 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:10.988847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:10.988850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:10.989237 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:10.989390 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:10.990522 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0811 03:11:10.990751 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 03:11:10.991742 12773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:54881
I20260811 03:11:11.004247 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 03:11:11 dist-test-slave-tm1g krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 03:11:11 dist-test-slave-tm1g krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:11.057790 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.042195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58492 (local address 127.11.60.62:42599)
0811 03:11:11.042474 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.042483 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.042786 (+ 303us) server_negotiation.cc:407] Connection header received
0811 03:11:11.043003 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.043008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.043082 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.043177 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:11.044629 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.045437 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.046819 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.047258 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.047369 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.051723 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:11.051729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:11.051732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:11.051775 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:11.054977 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:11.055690 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:11.055694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:11.055697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:11.055828 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:11.056302 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:11.056305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:11.056307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:11.056613 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:11.056828 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.057155 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.057362 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 03:11:11.062875 12774 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:11:11.070027 12774 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:11.098346 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.084234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.11.60.1:41505)
0811 03:11:11.085230 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.085240 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.086139 (+ 899us) server_negotiation.cc:407] Connection header received
0811 03:11:11.086534 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.086548 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.086880 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.087093 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:11.088290 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.090431 (+ 2141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.094222 (+ 3791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.096396 (+ 2174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.096535 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.096836 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.097799 (+ 963us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.098007 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":1010,"threads_started":1}
I20260811 03:11:11.099336 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.083560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:41505 (local address 127.0.0.1:45948)
0811 03:11:11.086001 (+ 2441us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.086024 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.086271 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.087454 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.087465 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.087517 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:11.088038 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.088054 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.090678 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.090686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.093971 (+ 3285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.093994 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.097092 (+ 3098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.097100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.097439 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.097526 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.098652 (+ 1126us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.098905 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2277,"thread_start_us":1030,"threads_started":1}
I20260811 03:11:11.103551 12930 tablet_service.cc:1467] Processing CreateTablet for tablet 7759033850a54203871d458e52dd12d1 (DEFAULT_TABLE table=test-table [id=f33ca5f5410343ff95ce9f2728f92feb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:11.105461 12930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7759033850a54203871d458e52dd12d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:11.105985 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.083541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:43895 (local address 127.0.0.1:48776)
0811 03:11:11.084358 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.084409 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.084663 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.098328 (+ 13665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.098333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.098427 (+ 94us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:11.098873 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.098887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.101027 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.101035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.103157 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.103172 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.105160 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.105166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.105366 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.105400 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.105485 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.105632 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":2067,"threads_started":1}
I20260811 03:11:11.106638 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.085375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:40795 (local address 127.0.0.1:46208)
0811 03:11:11.097875 (+ 12500us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.097895 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.098020 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.098895 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.098899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.098928 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:11.099339 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.099353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.101565 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.101570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.103581 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.103592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.105520 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.105529 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.105702 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.105725 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.105834 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.105958 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12389,"thread_start_us":12243,"threads_started":1}
I20260811 03:11:11.106645 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.085687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46208 (local address 127.11.60.2:40795)
0811 03:11:11.096648 (+ 10961us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.096663 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.098023 (+ 1360us) server_negotiation.cc:407] Connection header received
0811 03:11:11.098162 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.098170 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.098454 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.098671 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:11.099920 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.101403 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.103798 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.104990 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.105113 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.105368 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.105890 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.106284 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10733,"thread_start_us":10987,"threads_started":1}
I20260811 03:11:11.106921 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.084258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.11.60.3:43895)
0811 03:11:11.097604 (+ 13346us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.097615 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.097654 (+ 39us) server_negotiation.cc:407] Connection header received
0811 03:11:11.097763 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.097771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.098086 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.098356 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:11.099101 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.100894 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.103350 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.104901 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.106096 (+ 1195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.106357 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.106432 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.106591 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13166,"thread_start_us":12977,"threads_started":1}
I20260811 03:11:11.110733 13087 tablet_service.cc:1467] Processing CreateTablet for tablet 7759033850a54203871d458e52dd12d1 (DEFAULT_TABLE table=test-table [id=f33ca5f5410343ff95ce9f2728f92feb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:11.112196 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7759033850a54203871d458e52dd12d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:11.111907 13240 tablet_service.cc:1467] Processing CreateTablet for tablet 7759033850a54203871d458e52dd12d1 (DEFAULT_TABLE table=test-table [id=f33ca5f5410343ff95ce9f2728f92feb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:11.112994 13240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7759033850a54203871d458e52dd12d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:11.120739 13336 tablet_bootstrap.cc:492] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: Bootstrap starting.
I20260811 03:11:11.125689 13337 tablet_bootstrap.cc:492] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: Bootstrap starting.
I20260811 03:11:11.126168 13336 tablet_bootstrap.cc:654] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:11.127866 13336 log.cc:824] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:11.128131 13338 tablet_bootstrap.cc:492] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: Bootstrap starting.
I20260811 03:11:11.130020 13337 tablet_bootstrap.cc:654] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:11.132134 13338 tablet_bootstrap.cc:654] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:11.132781 13337 log.cc:824] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:11.133714 13338 log.cc:824] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:11.136112 13336 tablet_bootstrap.cc:492] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: No bootstrap required, opened a new log
I20260811 03:11:11.136278 13337 tablet_bootstrap.cc:492] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: No bootstrap required, opened a new log
I20260811 03:11:11.136452 13336 ts_tablet_manager.cc:1397] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260811 03:11:11.136570 13337 ts_tablet_manager.cc:1397] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 03:11:11.136744 13338 tablet_bootstrap.cc:492] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: No bootstrap required, opened a new log
I20260811 03:11:11.136996 13338 ts_tablet_manager.cc:1397] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260811 03:11:11.144999 13336 raft_consensus.cc:348] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } }
I20260811 03:11:11.145560 13336 raft_consensus.cc:374] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:11.145763 13336 raft_consensus.cc:729] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 209b6611c83a494b8cdbaef7d0d7ef51, State: Initialized, Role: FOLLOWER
I20260811 03:11:11.146579 13336 consensus_queue.cc:260] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } }
I20260811 03:11:11.148444 13336 ts_tablet_manager.cc:1428] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260811 03:11:11.149473 13337 raft_consensus.cc:348] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } }
I20260811 03:11:11.149950 13337 raft_consensus.cc:374] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:11.149696 13338 raft_consensus.cc:348] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } }
I20260811 03:11:11.150074 13337 raft_consensus.cc:729] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f33e4c458c949049f4a89d26cf15086, State: Initialized, Role: FOLLOWER
I20260811 03:11:11.150249 13338 raft_consensus.cc:374] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:11.150442 13338 raft_consensus.cc:729] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14679b0b266451b8fa59baea0068ccd, State: Initialized, Role: FOLLOWER
I20260811 03:11:11.150835 13337 consensus_queue.cc:260] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } }
I20260811 03:11:11.151059 13338 consensus_queue.cc:260] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } }
I20260811 03:11:11.152868 13165 heartbeater.cc:503] Master 127.11.60.62:42599 was elected leader, sending a full tablet report...
I20260811 03:11:11.154067 13338 ts_tablet_manager.cc:1428] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
I20260811 03:11:11.155253 13317 heartbeater.cc:503] Master 127.11.60.62:42599 was elected leader, sending a full tablet report...
I20260811 03:11:11.156013 13337 ts_tablet_manager.cc:1428] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: Time spent starting tablet: real 0.019s user 0.016s sys 0.004s
W20260811 03:11:11.170792 13011 tablet.cc:2369] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:11:11.203269 13166 tablet.cc:2369] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:11:11.237224 13318 tablet.cc:2369] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:11.283180 13344 raft_consensus.cc:482] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:11.283619 13344 raft_consensus.cc:504] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } }
I20260811 03:11:11.285773 13344 leader_election.cc:288] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f33e4c458c949049f4a89d26cf15086 (127.11.60.3:43895), 209b6611c83a494b8cdbaef7d0d7ef51 (127.11.60.1:41505)
I20260811 03:11:11.294199 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.286618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:44213 (local address 127.11.60.1:41505)
0811 03:11:11.286803 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.286808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.287539 (+ 731us) server_negotiation.cc:407] Connection header received
0811 03:11:11.287782 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.287788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.287941 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.288055 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:11.288791 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.289840 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.292074 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.293130 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.293238 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.293318 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.293790 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.293941 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 03:11:11.294585 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.286063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:41505 (local address 127.11.60.2:44213)
0811 03:11:11.287353 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.287377 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.287590 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.288110 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.288115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.288149 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:11.288583 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.288597 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.289982 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.289988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.291878 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.291892 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.293374 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.293380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.293569 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.293610 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.293729 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.293885 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"thread_start_us":776,"threads_started":1}
I20260811 03:11:11.294791 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.286066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:36971 (local address 127.11.60.3:43895)
0811 03:11:11.286396 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.286406 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.288072 (+ 1666us) server_negotiation.cc:407] Connection header received
0811 03:11:11.288269 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.288276 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.288364 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.288526 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:11.289410 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.290365 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.292644 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.293644 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.293770 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.293886 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.294289 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.294489 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 03:11:11.294888 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.286329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:43895 (local address 127.11.60.2:36971)
0811 03:11:11.287887 (+ 1558us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.287935 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.288112 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.288557 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.288561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.288582 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:11.289171 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.289188 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.290564 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.290572 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.292470 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.292487 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.293906 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.293913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.294082 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.294113 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.294220 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.294385 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1292,"thread_start_us":912,"threads_started":1}
I20260811 03:11:11.295871 13265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7759033850a54203871d458e52dd12d1" candidate_uuid: "b14679b0b266451b8fa59baea0068ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f33e4c458c949049f4a89d26cf15086" is_pre_election: true
I20260811 03:11:11.295837 12963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7759033850a54203871d458e52dd12d1" candidate_uuid: "b14679b0b266451b8fa59baea0068ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" is_pre_election: true
I20260811 03:11:11.296522 13265 raft_consensus.cc:2454] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b14679b0b266451b8fa59baea0068ccd in term 0.
I20260811 03:11:11.296589 12963 raft_consensus.cc:2454] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b14679b0b266451b8fa59baea0068ccd in term 0.
I20260811 03:11:11.297374 13033 leader_election.cc:302] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f33e4c458c949049f4a89d26cf15086, b14679b0b266451b8fa59baea0068ccd; no voters:
I20260811 03:11:11.297798 13344 raft_consensus.cc:2781] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:11.297981 13344 raft_consensus.cc:482] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:11.298103 13344 raft_consensus.cc:3037] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:11.300673 13344 raft_consensus.cc:504] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } }
I20260811 03:11:11.301482 13344 leader_election.cc:288] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [CANDIDATE]: Term 1 election: Requested vote from peers 4f33e4c458c949049f4a89d26cf15086 (127.11.60.3:43895), 209b6611c83a494b8cdbaef7d0d7ef51 (127.11.60.1:41505)
I20260811 03:11:11.301927 12963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7759033850a54203871d458e52dd12d1" candidate_uuid: "b14679b0b266451b8fa59baea0068ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "209b6611c83a494b8cdbaef7d0d7ef51"
I20260811 03:11:11.302065 13265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7759033850a54203871d458e52dd12d1" candidate_uuid: "b14679b0b266451b8fa59baea0068ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f33e4c458c949049f4a89d26cf15086"
I20260811 03:11:11.302167 12963 raft_consensus.cc:3037] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:11.302384 13265 raft_consensus.cc:3037] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:11.304847 12963 raft_consensus.cc:2454] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b14679b0b266451b8fa59baea0068ccd in term 1.
I20260811 03:11:11.305159 13265 raft_consensus.cc:2454] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b14679b0b266451b8fa59baea0068ccd in term 1.
I20260811 03:11:11.305450 13036 leader_election.cc:302] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 209b6611c83a494b8cdbaef7d0d7ef51, b14679b0b266451b8fa59baea0068ccd; no voters:
I20260811 03:11:11.305905 13344 raft_consensus.cc:2781] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:11.306593 13344 raft_consensus.cc:686] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 1 LEADER]: Becoming Leader. State: Replica: b14679b0b266451b8fa59baea0068ccd, State: Running, Role: LEADER
I20260811 03:11:11.307209 13344 consensus_queue.cc:237] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } }
I20260811 03:11:11.312991 12774 catalog_manager.cc:5495] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd reported cstate change: term changed from 0 to 1, leader changed from <none> to b14679b0b266451b8fa59baea0068ccd (127.11.60.2). New cstate: current_term: 1 leader_uuid: "b14679b0b266451b8fa59baea0068ccd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } health_report { overall_health: HEALTHY } } }
I20260811 03:11:11.354493 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.347535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46212 (local address 127.11.60.2:40795)
0811 03:11:11.347792 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.347799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.347823 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:11:11.348189 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.348193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.348280 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.348460 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:11.348985 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.349821 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.351777 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.352256 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.352418 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.353033 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:11.353679 (+ 646us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:11.353873 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.354022 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.354116 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 03:11:11.364961 13265 raft_consensus.cc:1261] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086 [term 1 FOLLOWER]: Refusing update from remote peer b14679b0b266451b8fa59baea0068ccd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:11.364974 12963 raft_consensus.cc:1261] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 FOLLOWER]: Refusing update from remote peer b14679b0b266451b8fa59baea0068ccd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:11.366153 13348 consensus_queue.cc:1035] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [LEADER]: Connected to new peer: Peer: permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:11.369155 13348 consensus_queue.cc:1035] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:11.385337 13352 mvcc.cc:205] Tried to move back new op lower bound from 7317167601098604544 to 7317167600879169536. Current Snapshot: MvccSnapshot[applied={T|T < 7317167601098604544}]
I20260811 03:11:11.403949 12774 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:11.404238 12774 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:11.408311 12774 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05: Sending DeleteTablet for 3 replicas of tablet 7759033850a54203871d458e52dd12d1
I20260811 03:11:11.409565 13240 tablet_service.cc:1514] Processing DeleteTablet for tablet 7759033850a54203871d458e52dd12d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:11 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:48776
I20260811 03:11:11.410081 12930 tablet_service.cc:1514] Processing DeleteTablet for tablet 7759033850a54203871d458e52dd12d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:11 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:45948
I20260811 03:11:11.410871 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet 7759033850a54203871d458e52dd12d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:11 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:46208
I20260811 03:11:11.411516 12774 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:11:11.413338 12774 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:11.415266 13360 tablet_replica.cc:330] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: stopping tablet replica
I20260811 03:11:11.415788 13360 raft_consensus.cc:2229] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:11.416360 13360 raft_consensus.cc:2258] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:11.421551 13360 ts_tablet_manager.cc:1905] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:11.422318 13086 tablet_service.cc:1467] Processing CreateTablet for tablet 0cfa900997e2435ea46a5a430d8f938e (DEFAULT_TABLE table=test-table [id=c7737434e5914eeba72d8fa20c05efda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:11.423025 13086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cfa900997e2435ea46a5a430d8f938e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:11.426937 12929 tablet_service.cc:1467] Processing CreateTablet for tablet 0cfa900997e2435ea46a5a430d8f938e (DEFAULT_TABLE table=test-table [id=c7737434e5914eeba72d8fa20c05efda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:11.428170 12929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cfa900997e2435ea46a5a430d8f938e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:11.428169 13362 tablet_replica.cc:330] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: stopping tablet replica
I20260811 03:11:11.428517 13361 tablet_replica.cc:330] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: stopping tablet replica
I20260811 03:11:11.428952 13361 raft_consensus.cc:2229] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:11.428834 13362 raft_consensus.cc:2229] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:11:11.429601 13361 raft_consensus.cc:2258] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:11.432727 13361 ts_tablet_manager.cc:1905] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:11.435791 13360 ts_tablet_manager.cc:1918] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:11.436028 13360 log.cc:1196] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/7759033850a54203871d458e52dd12d1
I20260811 03:11:11.436669 13360 ts_tablet_manager.cc:1939] T 7759033850a54203871d458e52dd12d1 P 4f33e4c458c949049f4a89d26cf15086: Deleting consensus metadata
I20260811 03:11:11.430770 13362 raft_consensus.cc:2258] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:11.438956 12757 catalog_manager.cc:4841] TS 4f33e4c458c949049f4a89d26cf15086 (127.11.60.3:43895): tablet 7759033850a54203871d458e52dd12d1 (table test-table [id=f33ca5f5410343ff95ce9f2728f92feb]) successfully deleted
I20260811 03:11:11.439881 13338 tablet_bootstrap.cc:492] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd: Bootstrap starting.
I20260811 03:11:11.440328 13240 tablet_service.cc:1467] Processing CreateTablet for tablet 0cfa900997e2435ea46a5a430d8f938e (DEFAULT_TABLE table=test-table [id=c7737434e5914eeba72d8fa20c05efda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:11.440629 13362 ts_tablet_manager.cc:1905] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:11.440814 13240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cfa900997e2435ea46a5a430d8f938e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:11.441939 13338 tablet_bootstrap.cc:654] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:11.442724 13361 ts_tablet_manager.cc:1918] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:11.443024 13361 log.cc:1196] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/7759033850a54203871d458e52dd12d1
I20260811 03:11:11.443892 13361 ts_tablet_manager.cc:1939] T 7759033850a54203871d458e52dd12d1 P 209b6611c83a494b8cdbaef7d0d7ef51: Deleting consensus metadata
I20260811 03:11:11.446349 12759 catalog_manager.cc:4841] TS 209b6611c83a494b8cdbaef7d0d7ef51 (127.11.60.1:41505): tablet 7759033850a54203871d458e52dd12d1 (table test-table [id=f33ca5f5410343ff95ce9f2728f92feb]) successfully deleted
I20260811 03:11:11.446668 13336 tablet_bootstrap.cc:492] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51: Bootstrap starting.
I20260811 03:11:11.448449 13338 tablet_bootstrap.cc:492] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd: No bootstrap required, opened a new log
I20260811 03:11:11.448594 13338 ts_tablet_manager.cc:1397] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 03:11:11.448740 13336 tablet_bootstrap.cc:654] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:11.451795 13337 tablet_bootstrap.cc:492] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086: Bootstrap starting.
I20260811 03:11:11.452857 13336 tablet_bootstrap.cc:492] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51: No bootstrap required, opened a new log
I20260811 03:11:11.453110 13336 ts_tablet_manager.cc:1397] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.001s
I20260811 03:11:11.454080 13337 tablet_bootstrap.cc:654] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:11.453977 13336 raft_consensus.cc:348] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.454382 13336 raft_consensus.cc:374] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:11.454550 13336 raft_consensus.cc:729] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 209b6611c83a494b8cdbaef7d0d7ef51, State: Initialized, Role: FOLLOWER
I20260811 03:11:11.454901 13336 consensus_queue.cc:260] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.455677 13336 ts_tablet_manager.cc:1428] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 03:11:11.461638 13338 raft_consensus.cc:348] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.461974 13338 raft_consensus.cc:374] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:11.462150 13338 raft_consensus.cc:729] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14679b0b266451b8fa59baea0068ccd, State: Initialized, Role: FOLLOWER
I20260811 03:11:11.463899 13338 consensus_queue.cc:260] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.466464 13337 tablet_bootstrap.cc:492] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086: No bootstrap required, opened a new log
I20260811 03:11:11.466626 13337 ts_tablet_manager.cc:1397] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.005s
I20260811 03:11:11.467437 13338 ts_tablet_manager.cc:1428] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260811 03:11:11.467528 13337 raft_consensus.cc:348] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.467756 13337 raft_consensus.cc:374] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:11.467849 13337 raft_consensus.cc:729] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f33e4c458c949049f4a89d26cf15086, State: Initialized, Role: FOLLOWER
I20260811 03:11:11.468140 13337 consensus_queue.cc:260] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.468858 13337 ts_tablet_manager.cc:1428] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 03:11:11.469647 13362 ts_tablet_manager.cc:1918] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:11.469919 13362 log.cc:1196] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/7759033850a54203871d458e52dd12d1
I20260811 03:11:11.470588 13362 ts_tablet_manager.cc:1939] T 7759033850a54203871d458e52dd12d1 P b14679b0b266451b8fa59baea0068ccd: Deleting consensus metadata
I20260811 03:11:11.472440 12759 catalog_manager.cc:4841] TS b14679b0b266451b8fa59baea0068ccd (127.11.60.2:40795): tablet 7759033850a54203871d458e52dd12d1 (table test-table [id=f33ca5f5410343ff95ce9f2728f92feb]) successfully deleted
I20260811 03:11:11.555711 13344 raft_consensus.cc:482] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:11.556042 13344 raft_consensus.cc:504] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.556777 13344 leader_election.cc:288] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 209b6611c83a494b8cdbaef7d0d7ef51 (127.11.60.1:41505), 4f33e4c458c949049f4a89d26cf15086 (127.11.60.3:43895)
I20260811 03:11:11.557399 13265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cfa900997e2435ea46a5a430d8f938e" candidate_uuid: "b14679b0b266451b8fa59baea0068ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f33e4c458c949049f4a89d26cf15086" is_pre_election: true
I20260811 03:11:11.557410 12963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cfa900997e2435ea46a5a430d8f938e" candidate_uuid: "b14679b0b266451b8fa59baea0068ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" is_pre_election: true
I20260811 03:11:11.557675 13265 raft_consensus.cc:2454] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b14679b0b266451b8fa59baea0068ccd in term 0.
I20260811 03:11:11.557683 12963 raft_consensus.cc:2454] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b14679b0b266451b8fa59baea0068ccd in term 0.
I20260811 03:11:11.558157 13036 leader_election.cc:302] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 209b6611c83a494b8cdbaef7d0d7ef51, b14679b0b266451b8fa59baea0068ccd; no voters:
I20260811 03:11:11.558475 13344 raft_consensus.cc:2781] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:11.558591 13344 raft_consensus.cc:482] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:11.558763 13344 raft_consensus.cc:3037] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:11.561224 13344 raft_consensus.cc:504] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.561942 13344 leader_election.cc:288] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [CANDIDATE]: Term 1 election: Requested vote from peers 209b6611c83a494b8cdbaef7d0d7ef51 (127.11.60.1:41505), 4f33e4c458c949049f4a89d26cf15086 (127.11.60.3:43895)
I20260811 03:11:11.562502 12963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cfa900997e2435ea46a5a430d8f938e" candidate_uuid: "b14679b0b266451b8fa59baea0068ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "209b6611c83a494b8cdbaef7d0d7ef51"
I20260811 03:11:11.562505 13265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cfa900997e2435ea46a5a430d8f938e" candidate_uuid: "b14679b0b266451b8fa59baea0068ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f33e4c458c949049f4a89d26cf15086"
I20260811 03:11:11.562736 12963 raft_consensus.cc:3037] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:11.562753 13265 raft_consensus.cc:3037] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:11.565348 12963 raft_consensus.cc:2454] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b14679b0b266451b8fa59baea0068ccd in term 1.
I20260811 03:11:11.565848 13265 raft_consensus.cc:2454] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b14679b0b266451b8fa59baea0068ccd in term 1.
I20260811 03:11:11.565861 13036 leader_election.cc:302] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 209b6611c83a494b8cdbaef7d0d7ef51, b14679b0b266451b8fa59baea0068ccd; no voters:
I20260811 03:11:11.566270 13344 raft_consensus.cc:2781] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:11.566565 13344 raft_consensus.cc:686] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 1 LEADER]: Becoming Leader. State: Replica: b14679b0b266451b8fa59baea0068ccd, State: Running, Role: LEADER
I20260811 03:11:11.566958 13344 consensus_queue.cc:237] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.570613 12774 catalog_manager.cc:5495] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd reported cstate change: term changed from 0 to 1, leader changed from <none> to b14679b0b266451b8fa59baea0068ccd (127.11.60.2). New cstate: current_term: 1 leader_uuid: "b14679b0b266451b8fa59baea0068ccd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } health_report { overall_health: UNKNOWN } } }
Aug 11 03:11:11 dist-test-slave-tm1g krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417869, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:11.621084 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.604678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42599 (local address 127.0.0.1:58496)
0811 03:11:11.605593 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.605622 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.605800 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.606198 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.606202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.606928 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:11.607141 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.607151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.608160 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.608168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.609172 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.609184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.610223 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.610228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.610424 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.611300 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:11.611317 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:11.614207 (+ 2890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:11.617907 (+ 3700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:11.617922 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:11.617930 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:11.618645 (+ 715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:11.619009 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:11.619014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:11.619017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:11.619277 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:11.619820 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:11.619828 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:11.620022 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.620179 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.620492 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":482,"threads_started":1}
I20260811 03:11:11.621130 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.604745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58496 (local address 127.11.60.62:42599)
0811 03:11:11.605939 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.605943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.605966 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:11:11.606006 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.606010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.606061 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.606176 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:11.607281 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.608032 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.609391 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.610063 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.610192 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.614446 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:11.614452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:11.614460 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:11.614489 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:11.617782 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:11.618763 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:11.618775 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:11.618778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:11.618895 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:11.619401 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:11.619406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:11.619409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:11.619688 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:11.619844 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.620266 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.620617 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":163,"server-negotiator.queue_time_us":1040,"thread_start_us":907,"threads_started":1}
I20260811 03:11:11.626789 12774 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:58496:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 03:11:11.628096 12774 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:11.635483 12929 tablet_service.cc:1467] Processing CreateTablet for tablet 8af35e591d314b36aa525241e8649ffd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b2304e45299e4271903260095a6a66e5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 03:11:11.635679 13240 tablet_service.cc:1467] Processing CreateTablet for tablet 8af35e591d314b36aa525241e8649ffd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b2304e45299e4271903260095a6a66e5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 03:11:11.636142 12929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af35e591d314b36aa525241e8649ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:11.636199 13240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af35e591d314b36aa525241e8649ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:11.639412 13086 tablet_service.cc:1467] Processing CreateTablet for tablet 8af35e591d314b36aa525241e8649ffd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b2304e45299e4271903260095a6a66e5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 03:11:11.639947 13086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af35e591d314b36aa525241e8649ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:11.644872 13337 tablet_bootstrap.cc:492] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: Bootstrap starting.
I20260811 03:11:11.648032 13337 tablet_bootstrap.cc:654] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:11.657152 13337 tablet_bootstrap.cc:492] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: No bootstrap required, opened a new log
I20260811 03:11:11.657292 13338 tablet_bootstrap.cc:492] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: Bootstrap starting.
I20260811 03:11:11.657487 13337 ts_tablet_manager.cc:1397] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260811 03:11:11.658593 13337 raft_consensus.cc:348] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.659022 13337 raft_consensus.cc:374] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:11.659219 13337 raft_consensus.cc:729] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f33e4c458c949049f4a89d26cf15086, State: Initialized, Role: FOLLOWER
I20260811 03:11:11.659696 13338 tablet_bootstrap.cc:654] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:11.659708 13337 consensus_queue.cc:260] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.660540 13343 tablet_replica.cc:439] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } }
I20260811 03:11:11.660912 13343 tablet_replica.cc:442] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:11.660714 13337 ts_tablet_manager.cc:1428] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 03:11:11.662413 13336 tablet_bootstrap.cc:492] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: Bootstrap starting.
I20260811 03:11:11.663285 13338 tablet_bootstrap.cc:492] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: No bootstrap required, opened a new log
I20260811 03:11:11.663538 13338 ts_tablet_manager.cc:1397] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 03:11:11.664539 13336 tablet_bootstrap.cc:654] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:11.664932 13338 raft_consensus.cc:348] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.665292 13338 raft_consensus.cc:374] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:11.665480 13338 raft_consensus.cc:729] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14679b0b266451b8fa59baea0068ccd, State: Initialized, Role: FOLLOWER
I20260811 03:11:11.665827 13338 consensus_queue.cc:260] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.666673 13338 ts_tablet_manager.cc:1428] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 03:11:11.666852 13344 tablet_replica.cc:439] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } }
I20260811 03:11:11.667249 13344 tablet_replica.cc:442] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:11.669507 13336 tablet_bootstrap.cc:492] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: No bootstrap required, opened a new log
I20260811 03:11:11.669690 13336 ts_tablet_manager.cc:1397] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 03:11:11.670689 13336 raft_consensus.cc:348] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.671036 13336 raft_consensus.cc:374] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:11.671172 13336 raft_consensus.cc:729] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 209b6611c83a494b8cdbaef7d0d7ef51, State: Initialized, Role: FOLLOWER
I20260811 03:11:11.671507 13336 consensus_queue.cc:260] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.672019 13342 tablet_replica.cc:439] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } }
I20260811 03:11:11.672394 13342 tablet_replica.cc:442] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:11.673034 13336 ts_tablet_manager.cc:1428] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:11.685907 13342 raft_consensus.cc:482] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:11.686125 13342 raft_consensus.cc:504] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.688172 13342 leader_election.cc:288] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b14679b0b266451b8fa59baea0068ccd (127.11.60.2:40795), 4f33e4c458c949049f4a89d26cf15086 (127.11.60.3:43895)
I20260811 03:11:11.698096 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.689651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:49645 (local address 127.11.60.2:40795)
0811 03:11:11.689841 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.689848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.689873 (+ 25us) server_negotiation.cc:407] Connection header received
0811 03:11:11.689931 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.689935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.690049 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.690143 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:11.691097 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.692307 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.695299 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.696636 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.696740 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.696830 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.697634 (+ 804us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.697776 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 03:11:11.698086 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.689557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:48323 (local address 127.11.60.3:43895)
0811 03:11:11.689879 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.689885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.689907 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:11:11.689983 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.689989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.690082 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.690184 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:11.691422 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.692492 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.695300 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.696594 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.696714 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.696815 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.697634 (+ 819us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.697775 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260811 03:11:11.699256 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.688965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:43895 (local address 127.11.60.1:48323)
0811 03:11:11.689622 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.689645 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.689759 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.690747 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.690751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.690785 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:11.691217 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.691231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.692650 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.692655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.695104 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.695120 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.697117 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.697122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.697334 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.697471 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.698279 (+ 808us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.698476 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":113,"thread_start_us":1495,"threads_started":1}
I20260811 03:11:11.699734 13380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.688385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:40795 (local address 127.11.60.1:49645)
0811 03:11:11.688990 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.689007 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.689129 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.690368 (+ 1239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.690373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.690402 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:11.690868 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.690885 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.692510 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.692514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.695104 (+ 2590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.695120 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.697106 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.697113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.697335 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.697472 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.698703 (+ 1231us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.698868 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":158,"thread_start_us":1947,"threads_started":1}
I20260811 03:11:11.700062 13265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af35e591d314b36aa525241e8649ffd" candidate_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f33e4c458c949049f4a89d26cf15086" is_pre_election: true
I20260811 03:11:11.700409 13265 raft_consensus.cc:2454] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 209b6611c83a494b8cdbaef7d0d7ef51 in term 0.
I20260811 03:11:11.701006 12874 leader_election.cc:302] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 209b6611c83a494b8cdbaef7d0d7ef51, 4f33e4c458c949049f4a89d26cf15086; no voters:
I20260811 03:11:11.701049 13116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af35e591d314b36aa525241e8649ffd" candidate_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b14679b0b266451b8fa59baea0068ccd" is_pre_election: true
I20260811 03:11:11.701495 13342 raft_consensus.cc:2781] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:11.701643 13342 raft_consensus.cc:482] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:11.701774 13342 raft_consensus.cc:3037] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:11.701587 13116 raft_consensus.cc:2454] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 209b6611c83a494b8cdbaef7d0d7ef51 in term 0.
I20260811 03:11:11.705129 13342 raft_consensus.cc:504] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.706171 13342 leader_election.cc:288] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [CANDIDATE]: Term 1 election: Requested vote from peers b14679b0b266451b8fa59baea0068ccd (127.11.60.2:40795), 4f33e4c458c949049f4a89d26cf15086 (127.11.60.3:43895)
I20260811 03:11:11.706441 13265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af35e591d314b36aa525241e8649ffd" candidate_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f33e4c458c949049f4a89d26cf15086"
I20260811 03:11:11.706590 13116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af35e591d314b36aa525241e8649ffd" candidate_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b14679b0b266451b8fa59baea0068ccd"
I20260811 03:11:11.706700 13265 raft_consensus.cc:3037] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:11.706864 13116 raft_consensus.cc:3037] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:11.709892 13116 raft_consensus.cc:2454] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 209b6611c83a494b8cdbaef7d0d7ef51 in term 1.
I20260811 03:11:11.710211 13265 raft_consensus.cc:2454] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 209b6611c83a494b8cdbaef7d0d7ef51 in term 1.
I20260811 03:11:11.710678 12876 leader_election.cc:302] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 209b6611c83a494b8cdbaef7d0d7ef51, b14679b0b266451b8fa59baea0068ccd; no voters:
I20260811 03:11:11.711158 13342 raft_consensus.cc:2781] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:11.722556 13342 raft_consensus.cc:686] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 LEADER]: Becoming Leader. State: Replica: 209b6611c83a494b8cdbaef7d0d7ef51, State: Running, Role: LEADER
I20260811 03:11:11.723145 13342 consensus_queue.cc:237] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } }
I20260811 03:11:11.725375 13383 tablet_replica.cc:439] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: TxnStatusTablet state changed. Reason: New leader 209b6611c83a494b8cdbaef7d0d7ef51. Latest consensus state: current_term: 1 leader_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } }
I20260811 03:11:11.726104 13383 tablet_replica.cc:442] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: This TxnStatusTablet replica's current role is: LEADER
I20260811 03:11:11.727314 13385 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 03:11:11.727644 13385 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 03:11:11.727665 12773 catalog_manager.cc:5495] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 209b6611c83a494b8cdbaef7d0d7ef51 (127.11.60.1). New cstate: current_term: 1 leader_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } health_report { overall_health: UNKNOWN } } }
I20260811 03:11:11.765461 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.756846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45954 (local address 127.11.60.1:41505)
0811 03:11:11.757054 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.757060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.757090 (+ 30us) server_negotiation.cc:407] Connection header received
0811 03:11:11.757340 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.757346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.757465 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.757630 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:11.758334 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.759585 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.761798 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.762639 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.762773 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.763562 (+ 789us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:11.764492 (+ 930us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:11.764650 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.764887 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.764987 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 03:11:11.765466 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.756656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:41505 (local address 127.0.0.1:45954)
0811 03:11:11.756912 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.756935 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.757103 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.757686 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.757692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.757705 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:11:11.758121 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.758147 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.759765 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.759771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.761628 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.761644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.762966 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.762974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.763229 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.763312 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:11:11.764631 (+ 1319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:11:11.764665 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.764780 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.764999 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 03:11:11.771476 13265 raft_consensus.cc:1261] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086 [term 1 FOLLOWER]: Refusing update from remote peer 209b6611c83a494b8cdbaef7d0d7ef51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:11.772424 13116 raft_consensus.cc:1261] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd [term 1 FOLLOWER]: Refusing update from remote peer 209b6611c83a494b8cdbaef7d0d7ef51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:11.772541 13383 consensus_queue.cc:1035] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:11.773662 13383 consensus_queue.cc:1035] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:11.780180 13383 tablet_replica.cc:439] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } }
I20260811 03:11:11.780484 13383 tablet_replica.cc:442] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: This TxnStatusTablet replica's current role is: LEADER
I20260811 03:11:11.781694 13343 tablet_replica.cc:439] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: TxnStatusTablet state changed. Reason: New leader 209b6611c83a494b8cdbaef7d0d7ef51. Latest consensus state: current_term: 1 leader_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } }
I20260811 03:11:11.781978 13343 tablet_replica.cc:442] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:11.784058 13354 mvcc.cc:205] Tried to move back new op lower bound from 7317167602766856192 to 7317167602581815296. Current Snapshot: MvccSnapshot[applied={T|T < 7317167602766856192}]
I20260811 03:11:11.784415 13343 tablet_replica.cc:439] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } }
I20260811 03:11:11.784642 13343 tablet_replica.cc:442] T 8af35e591d314b36aa525241e8649ffd P 4f33e4c458c949049f4a89d26cf15086: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:11.787405 13344 tablet_replica.cc:439] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: TxnStatusTablet state changed. Reason: New leader 209b6611c83a494b8cdbaef7d0d7ef51. Latest consensus state: current_term: 1 leader_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } }
I20260811 03:11:11.787914 13344 tablet_replica.cc:442] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:11.788569 13342 tablet_replica.cc:439] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } }
I20260811 03:11:11.788996 13342 tablet_replica.cc:442] T 8af35e591d314b36aa525241e8649ffd P 209b6611c83a494b8cdbaef7d0d7ef51: This TxnStatusTablet replica's current role is: LEADER
I20260811 03:11:11.792006 13344 tablet_replica.cc:439] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 } } peers { permanent_uuid: "b14679b0b266451b8fa59baea0068ccd" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40795 } } peers { permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 } } }
I20260811 03:11:11.792337 13344 tablet_replica.cc:442] T 8af35e591d314b36aa525241e8649ffd P b14679b0b266451b8fa59baea0068ccd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:11.792544 13351 mvcc.cc:205] Tried to move back new op lower bound from 7317167602766856192 to 7317167602581815296. Current Snapshot: MvccSnapshot[applied={T|T < 7317167602766856192 or (T in {7317167602766856192})}]
I20260811 03:11:11.826822 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.815870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:34537 (local address 127.11.60.1:41505)
0811 03:11:11.816100 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.816107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.817257 (+ 1150us) server_negotiation.cc:407] Connection header received
0811 03:11:11.817590 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.817598 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.817708 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.817844 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:11.818900 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.820519 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.823462 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.824485 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.824629 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.825458 (+ 829us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:11.825663 (+ 205us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:11.825761 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.826046 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.826163 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 03:11:11.827137 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.815725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:41505 (local address 127.11.60.2:34537)
0811 03:11:11.817113 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.817136 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.817415 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.818067 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.818072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.818097 (+ 25us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:11:11.818513 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.818535 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.820693 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.820699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.823265 (+ 2566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.823282 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.824795 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.824807 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.825109 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.825204 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:11:11.825830 (+ 626us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:11:11.825850 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.825945 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.826136 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1149,"thread_start_us":773,"threads_started":1}
I20260811 03:11:11.843307 12963 raft_consensus.cc:1261] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 FOLLOWER]: Refusing update from remote peer b14679b0b266451b8fa59baea0068ccd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:11.844072 13265 raft_consensus.cc:1261] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086 [term 1 FOLLOWER]: Refusing update from remote peer b14679b0b266451b8fa59baea0068ccd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:11.844887 13348 consensus_queue.cc:1035] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f33e4c458c949049f4a89d26cf15086" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:11.846186 13348 consensus_queue.cc:1035] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [LEADER]: Connected to new peer: Peer: permanent_uuid: "209b6611c83a494b8cdbaef7d0d7ef51" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:11.909060 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.895443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:37515 (local address 127.11.60.2:40795)
0811 03:11:11.895624 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:11.895631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:11.899305 (+ 3674us) server_negotiation.cc:407] Connection header received
0811 03:11:11.899587 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:11.899593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:11.899681 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:11.899776 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:11.900814 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.902211 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.905136 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.906167 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.906290 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.908073 (+ 1783us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:11.908242 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:11.908326 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:11.908621 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 03:11:11.908733 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 03:11:11.909281 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:11.895783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:40795 (local address 127.11.60.1:37515)
0811 03:11:11.899151 (+ 3368us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:11.899190 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:11.899366 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:11.899980 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:11.899985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:11.900000 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:11:11.900613 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.900627 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.902569 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.902575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.904963 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:11.904980 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:11.906917 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:11.906928 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:11.907288 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:11.907817 (+ 529us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:11:11.908427 (+ 610us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:11:11.908448 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 03:11:11.908568 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 03:11:11.908732 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3067,"thread_start_us":2879,"threads_started":1}
I20260811 03:11:11.975821 12773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:11.976132 12773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:11.979319 12773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44bf6f1690e14f3e91ccbe14ac7c8e05: Sending DeleteTablet for 3 replicas of tablet 0cfa900997e2435ea46a5a430d8f938e
I20260811 03:11:11.980376 13240 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cfa900997e2435ea46a5a430d8f938e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:11 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:48776
I20260811 03:11:11.980489 12929 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cfa900997e2435ea46a5a430d8f938e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:11 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:45948
I20260811 03:11:11.980980 13086 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cfa900997e2435ea46a5a430d8f938e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:11 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:46208
I20260811 03:11:12.005417 13401 tablet_replica.cc:330] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51: stopping tablet replica
I20260811 03:11:12.005716 13401 raft_consensus.cc:2229] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:12.006048 13401 raft_consensus.cc:2258] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:12.009992 13400 tablet_replica.cc:330] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086: stopping tablet replica
I20260811 03:11:12.010228 13400 raft_consensus.cc:2229] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:12.011889 13401 ts_tablet_manager.cc:1905] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:12.012959 13403 tablet_replica.cc:330] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd: stopping tablet replica
I20260811 03:11:12.013177 13400 raft_consensus.cc:2258] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:12.014442 13400 ts_tablet_manager.cc:1905] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:12.014854 13403 raft_consensus.cc:2229] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:11:12.015347 13403 raft_consensus.cc:2258] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:12.019613 13403 ts_tablet_manager.cc:1905] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:12.029791 13401 ts_tablet_manager.cc:1918] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:11:12.029963 13401 log.cc:1196] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/0cfa900997e2435ea46a5a430d8f938e
I20260811 03:11:12.030542 13401 ts_tablet_manager.cc:1939] T 0cfa900997e2435ea46a5a430d8f938e P 209b6611c83a494b8cdbaef7d0d7ef51: Deleting consensus metadata
I20260811 03:11:12.031924 12759 catalog_manager.cc:4841] TS 209b6611c83a494b8cdbaef7d0d7ef51 (127.11.60.1:41505): tablet 0cfa900997e2435ea46a5a430d8f938e (table test-table [id=c7737434e5914eeba72d8fa20c05efda]) successfully deleted
I20260811 03:11:12.032683 13400 ts_tablet_manager.cc:1918] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:11:12.032837 13400 log.cc:1196] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/0cfa900997e2435ea46a5a430d8f938e
I20260811 03:11:12.033402 13400 ts_tablet_manager.cc:1939] T 0cfa900997e2435ea46a5a430d8f938e P 4f33e4c458c949049f4a89d26cf15086: Deleting consensus metadata
I20260811 03:11:12.034389 13403 ts_tablet_manager.cc:1918] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:11:12.034600 12757 catalog_manager.cc:4841] TS 4f33e4c458c949049f4a89d26cf15086 (127.11.60.3:43895): tablet 0cfa900997e2435ea46a5a430d8f938e (table test-table [id=c7737434e5914eeba72d8fa20c05efda]) successfully deleted
Aug 11 03:11:12 dist-test-slave-tm1g krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:12.038722 13403 log.cc:1196] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/0cfa900997e2435ea46a5a430d8f938e
I20260811 03:11:12.039449 13403 ts_tablet_manager.cc:1939] T 0cfa900997e2435ea46a5a430d8f938e P b14679b0b266451b8fa59baea0068ccd: Deleting consensus metadata
I20260811 03:11:12.040715 12759 catalog_manager.cc:4841] TS b14679b0b266451b8fa59baea0068ccd (127.11.60.2:40795): tablet 0cfa900997e2435ea46a5a430d8f938e (table test-table [id=c7737434e5914eeba72d8fa20c05efda]) successfully deleted
I20260811 03:11:12.051717 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:12.004025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45956 (local address 127.11.60.1:41505)
0811 03:11:12.004228 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:12.004237 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:12.012730 (+ 8493us) server_negotiation.cc:407] Connection header received
0811 03:11:12.014157 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:12.014163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:12.014257 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:12.014368 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:12.023127 (+ 8759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:12.024356 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:12.032412 (+ 8056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:12.033335 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:12.033451 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:12.041250 (+ 7799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:12.041269 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:12.041290 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:12.041343 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:12.046686 (+ 5343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:12.047933 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:12.047944 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:12.047948 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:12.048202 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:12.048757 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:12.048764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:12.048770 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:12.049443 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:12.050008 (+ 565us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:12.050970 (+ 962us) server_negotiation.cc:299] Negotiation successful
0811 03:11:12.051283 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 03:11:12.054044 13004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45956
I20260811 03:11:12.086442 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:12.063717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58498 (local address 127.11.60.62:42599)
0811 03:11:12.064027 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:12.064033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:12.066859 (+ 2826us) server_negotiation.cc:407] Connection header received
0811 03:11:12.067159 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:12.067167 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:12.067270 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:12.067356 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:12.069794 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:12.071238 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:12.073401 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:12.074440 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:12.074574 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:12.078791 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:12.078797 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:12.078803 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:12.078848 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:12.082994 (+ 4146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:12.083858 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:12.083864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:12.083868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:12.084078 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:12.084515 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:12.084521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:12.084524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:12.084947 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:12.085177 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:12.085575 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 03:11:12.085843 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
W20260811 03:11:12.087152 12773 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58498
I20260811 03:11:12.089138 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 12854
I20260811 03:11:12.106032 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:12.091351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45972 (local address 127.11.60.1:41505)
0811 03:11:12.091549 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:12.091553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:12.091981 (+ 428us) server_negotiation.cc:407] Connection header received
0811 03:11:12.092288 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:12.092293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:12.092368 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:12.092476 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:12.093694 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:12.094559 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:12.095763 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:12.096230 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:12.096369 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:12.099155 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:12.099163 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:12.099172 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:12.099206 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:12.102815 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:12.103525 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:12.103532 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:12.103535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:12.103683 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:12.104184 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:12.104190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:12.104193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:12.104566 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:12.104950 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:12.105249 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 03:11:12.105615 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 03:11:12.106761 13004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45972
W20260811 03:11:12.107472 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 12854: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:11:12.107712 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 12854
I20260811 03:11:12.132623 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13014
Aug 11 03:11:12 dist-test-slave-tm1g krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:11:12.147991 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:12.133391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46216 (local address 127.11.60.2:40795)
0811 03:11:12.133632 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:12.133638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:12.133657 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:11:12.133876 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:12.133882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:12.133960 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:12.134079 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:12.135353 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:12.136117 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:12.137372 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:12.137834 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:12.137976 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:12.142119 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:12.142133 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:12.142149 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:12.142180 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:12.144979 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:12.145653 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:12.145663 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:12.145669 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:12.145813 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:12.146188 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:12.146193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:12.146197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:12.146814 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:12.147141 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:12.147406 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 03:11:12.147660 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260811 03:11:12.148778 13159 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46216
W20260811 03:11:12.149418 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13014: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:11:12.149708 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 13014
I20260811 03:11:12.176805 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13169
Aug 11 03:11:12 dist-test-slave-tm1g krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:11:12.192992 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:12.177648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.11.60.3:43895)
0811 03:11:12.177848 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:12.177854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:12.177877 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:11:12.178037 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:12.178042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:12.178119 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:12.178208 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:12.179664 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:12.180393 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:12.181637 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:12.182125 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:12.182294 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:12.186770 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:12.186787 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:12.186806 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:12.186870 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:12.189883 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:12.190810 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:12.190818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:12.190820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:12.190976 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:12.191410 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:12.191414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:12.191416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:12.191783 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:12.192123 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:12.192424 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 03:11:12.192669 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 03:11:12.193643 13312 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48792
W20260811 03:11:12.194250 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13169: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:11:12.194433 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 13169
I20260811 03:11:12.214969 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 12736
I20260811 03:11:12.216167 12819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:12.332315 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 12736
2026-08-11T03:11:12Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4897 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:11:12 dist-test-slave-tm1g krb5kdc[13412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:11:12 dist-test-slave-tm1g krb5kdc[13412](info): set up 2 sockets
Aug 11 03:11:12 dist-test-slave-tm1g krb5kdc[13412](info): commencing operation
krb5kdc: starting...
W20260811 03:11:13.478420 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:11:13 dist-test-slave-tm1g krb5kdc[13412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:11:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:11:13.684162 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:46435
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:46435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:14.015497 13425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:14.016078 13425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:14.016455 13425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:14.023985 13425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:14.024088 13425 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 03:11:14.024192 13425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:14.024327 13425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:14.024415 13425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:14.024528 13425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:14.034739 13425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:46435
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:46435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:14.036154 13425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:14.037539 13425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:14.049935 13431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:14.050060 13430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:14.050043 13433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:14.050956 13425 server_base.cc:1034] running on GCE node
I20260811 03:11:14.051745 13425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:14.053175 13425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:14.054450 13425 hybrid_clock.cc:648] HybridClock initialized: now 1786417874054360 us; error 43 us; skew 500 ppm
I20260811 03:11:14.054785 13425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:14 dist-test-slave-tm1g krb5kdc[13412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:14.057971 13425 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:11:14.059593 13425 webserver.cc:460] Webserver started at http://127.11.60.62:41063/ using document root <none> and password file <none>
I20260811 03:11:14.060315 13425 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:14.060487 13425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:14.060817 13425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:14.063838 13425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "38c0d65d6541405286c0d25d485911df"
format_stamp: "Formatted at 2026-08-11 03:11:14 on dist-test-slave-tm1g"
server_key: "c6f0f6b355ae837db3b1ebd2e7cb7348"
server_key_iv: "8d7f8224a533e5df7fabb6881b2f1f4b"
server_key_version: "encryptionkey@0"
I20260811 03:11:14.064613 13425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "38c0d65d6541405286c0d25d485911df"
format_stamp: "Formatted at 2026-08-11 03:11:14 on dist-test-slave-tm1g"
server_key: "c6f0f6b355ae837db3b1ebd2e7cb7348"
server_key_iv: "8d7f8224a533e5df7fabb6881b2f1f4b"
server_key_version: "encryptionkey@0"
I20260811 03:11:14.069912 13425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:11:14.073297 13440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:14.074714 13425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:14.074896 13425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "38c0d65d6541405286c0d25d485911df"
format_stamp: "Formatted at 2026-08-11 03:11:14 on dist-test-slave-tm1g"
server_key: "c6f0f6b355ae837db3b1ebd2e7cb7348"
server_key_iv: "8d7f8224a533e5df7fabb6881b2f1f4b"
server_key_version: "encryptionkey@0"
I20260811 03:11:14.075119 13425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:14.093971 13425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:14.098163 13425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:14.098479 13425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:14.121110 13425 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:46435
I20260811 03:11:14.121132 13507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:46435 every 8 connection(s)
I20260811 03:11:14.122572 13425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:14.123955 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 13425
I20260811 03:11:14.124190 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:14.125056 11504 external_mini_cluster.cc:1428] Setting key ecdadc997f84a957999bc1f8cde15962
I20260811 03:11:14.127504 13508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:14.138718 13508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df: Bootstrap starting.
Aug 11 03:11:14 dist-test-slave-tm1g krb5kdc[13412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:14.142491 13508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:14.143573 13508 log.cc:824] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:14.146960 13508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df: No bootstrap required, opened a new log
I20260811 03:11:14.147765 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:14.127226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.11.60.62:46435)
0811 03:11:14.128075 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:14.128087 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:14.128128 (+ 41us) server_negotiation.cc:407] Connection header received
0811 03:11:14.129737 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:14.129756 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:14.130143 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:14.130588 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:14.132067 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:14.133391 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:14.134713 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:14.135417 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:14.135618 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:14.140061 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:14.140069 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:14.140082 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:14.140125 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:14.143472 (+ 3347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:14.144387 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:14.144396 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:14.144404 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:14.144623 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:14.145016 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:14.145020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:14.145022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:14.145538 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:14.145748 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:14.146240 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 03:11:14.146653 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":435,"threads_started":1}
I20260811 03:11:14.154471 13508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c0d65d6541405286c0d25d485911df" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46435 } }
I20260811 03:11:14.154841 13508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:14.154968 13508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c0d65d6541405286c0d25d485911df, State: Initialized, Role: FOLLOWER
I20260811 03:11:14.155583 13508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c0d65d6541405286c0d25d485911df" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46435 } }
I20260811 03:11:14.155817 13508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:14.155951 13508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:14.156132 13508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:14.158386 13508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c0d65d6541405286c0d25d485911df" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46435 } }
I20260811 03:11:14.158896 13508 leader_election.cc:302] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38c0d65d6541405286c0d25d485911df; no voters:
I20260811 03:11:14.159463 13508 leader_election.cc:288] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:14.159605 13513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:14.160324 13513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [term 1 LEADER]: Becoming Leader. State: Replica: 38c0d65d6541405286c0d25d485911df, State: Running, Role: LEADER
I20260811 03:11:14.160835 13513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c0d65d6541405286c0d25d485911df" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46435 } }
I20260811 03:11:14.161552 13508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:14.164007 13515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38c0d65d6541405286c0d25d485911df. Latest consensus state: current_term: 1 leader_uuid: "38c0d65d6541405286c0d25d485911df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c0d65d6541405286c0d25d485911df" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46435 } } }
I20260811 03:11:14.164428 13515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:14.164347 13514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38c0d65d6541405286c0d25d485911df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c0d65d6541405286c0d25d485911df" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46435 } } }
I20260811 03:11:14.164646 13514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:14.166792 13519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:14.172402 13519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:14.193089 13519 catalog_manager.cc:1347] Generated new cluster ID: 5e8e5591cb4742b58ebaac777308a711
I20260811 03:11:14.193235 13519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:14.218403 13519 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:14.220446 13519 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:14.231740 13519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df: Generated new TSK 0
I20260811 03:11:14.232822 13519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:11:14.290253 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:46435
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:14.624794 13536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:14.625147 13536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:14.625341 13536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:14.632715 13536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:14.632881 13536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:14.633029 13536 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 03:11:14.633126 13536 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 03:11:14.633260 13536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:14.643352 13536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.60.62:46435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:14.644847 13536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:14.646263 13536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:14.659420 13541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:14.659448 13542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:14.659689 13544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:14.660219 13536 server_base.cc:1034] running on GCE node
I20260811 03:11:14.660914 13536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:14.661906 13536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:14.663133 13536 hybrid_clock.cc:648] HybridClock initialized: now 1786417874663098 us; error 59 us; skew 500 ppm
I20260811 03:11:14.663455 13536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:14 dist-test-slave-tm1g krb5kdc[13412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:14.666735 13536 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:11:14.668107 13536 webserver.cc:460] Webserver started at http://127.11.60.1:41539/ using document root <none> and password file <none>
I20260811 03:11:14.668780 13536 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:14.668910 13536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:14.669227 13536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:14.672196 13536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "f699cf80b4f94fb0baae8cbf8e308269"
format_stamp: "Formatted at 2026-08-11 03:11:14 on dist-test-slave-tm1g"
server_key: "1e3da66aef2e08add393ac3ab492e9e4"
server_key_iv: "122b9244b74ccfe3701ca99d12815ce4"
server_key_version: "encryptionkey@0"
I20260811 03:11:14.672967 13536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "f699cf80b4f94fb0baae8cbf8e308269"
format_stamp: "Formatted at 2026-08-11 03:11:14 on dist-test-slave-tm1g"
server_key: "1e3da66aef2e08add393ac3ab492e9e4"
server_key_iv: "122b9244b74ccfe3701ca99d12815ce4"
server_key_version: "encryptionkey@0"
I20260811 03:11:14.678086 13536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:11:14.681622 13551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:14.683027 13536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:14.683229 13536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "f699cf80b4f94fb0baae8cbf8e308269"
format_stamp: "Formatted at 2026-08-11 03:11:14 on dist-test-slave-tm1g"
server_key: "1e3da66aef2e08add393ac3ab492e9e4"
server_key_iv: "122b9244b74ccfe3701ca99d12815ce4"
server_key_version: "encryptionkey@0"
I20260811 03:11:14.683423 13536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:14.702740 13536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:14.707379 13536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:14.707700 13536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:14.710578 13536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:14.710712 13536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:14.710857 13536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:14.710925 13536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 03:11:14 dist-test-slave-tm1g krb5kdc[13412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:14.734328 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:14.710970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:47123 (local address 127.11.60.62:46435)
0811 03:11:14.711970 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:14.711979 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:14.712829 (+ 850us) server_negotiation.cc:407] Connection header received
0811 03:11:14.715042 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:14.715048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:14.715160 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:14.715317 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:14.717844 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:14.718943 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:14.721681 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:14.722109 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:14.722248 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:14.727473 (+ 5225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:14.727482 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:14.727490 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:14.727541 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:14.730946 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:14.731836 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:14.731843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:14.731847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:14.731991 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:14.732463 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:14.732469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:14.732473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:14.732764 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:14.732922 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:14.733642 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 03:11:14.733872 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":744,"threads_started":1}
I20260811 03:11:14.735896 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:14.711655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:46435 (local address 127.11.60.1:47123)
0811 03:11:14.712629 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:14.712679 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:14.714738 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:14.715615 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:14.715628 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:14.716496 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:14.717549 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:14.717583 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:14.719128 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:14.719132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:14.721525 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:14.721539 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:14.722407 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:14.722419 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:14.722821 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:14.723724 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:14.723763 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:14.727213 (+ 3450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:14.731116 (+ 3903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:14.731125 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:14.731142 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:14.731708 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:14.732121 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:14.732126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:14.732130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:14.732346 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:14.732905 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:14.732913 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:14.733280 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 03:11:14.733565 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 03:11:14.734100 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":407,"threads_started":1}
I20260811 03:11:14.759061 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:14.743062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:56683 (local address 127.11.60.62:46435)
0811 03:11:14.743249 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:14.743255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:14.743655 (+ 400us) server_negotiation.cc:407] Connection header received
0811 03:11:14.743855 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:14.743859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:14.743921 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:14.743999 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:14.745511 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:14.746417 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:14.748655 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:14.749320 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:14.749475 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:14.752422 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:14.752426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:14.752429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:14.752461 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:14.755521 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:14.756559 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:14.756566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:14.756569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:14.756743 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:14.757206 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:14.757212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:14.757216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:14.757642 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:14.757806 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:14.758469 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 03:11:14.758719 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 03:11:14.759161 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:14.742642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:46435 (local address 127.11.60.1:56683)
0811 03:11:14.743525 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:14.743551 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:14.743731 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:14.744069 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:14.744077 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:14.744850 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:14.745342 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:14.745359 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:14.746595 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:14.746602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:14.748504 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:14.748518 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:14.749571 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:14.749586 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:14.749869 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:14.750772 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:14.750804 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:14.752151 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:14.755662 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:14.755667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:14.755675 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:14.756319 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:14.756888 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:14.756898 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:14.756903 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:14.757083 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:14.757754 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:14.757759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:14.758022 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 03:11:14.758220 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 03:11:14.758628 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":615,"threads_started":1}
I20260811 03:11:14.765580 13536 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:37523
I20260811 03:11:14.765666 13682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:37523 every 8 connection(s)
I20260811 03:11:14.766885 13536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:14.768860 13683 heartbeater.cc:347] Connected to a master server at 127.11.60.62:46435
I20260811 03:11:14.769101 13683 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:14.769769 13683 heartbeater.cc:511] Master 127.11.60.62:46435 requested a full tablet report, sending...
I20260811 03:11:14.771689 13460 ts_manager.cc:194] Registered new tserver with Master: f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523)
I20260811 03:11:14.772902 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 13536
I20260811 03:11:14.773111 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:14.773694 13460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:47123
I20260811 03:11:14.773919 11504 external_mini_cluster.cc:1428] Setting key 34178c40c5042287f9b986109eb8c3ce
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:11:14.822100 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:46435
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:15.158041 13687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:15.158427 13687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:15.158638 13687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:15.166096 13687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:15.166195 13687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:15.166388 13687 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 03:11:15.166460 13687 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 03:11:15.166587 13687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:15.176510 13687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.60.62:46435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:15.177924 13687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:15.179508 13687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:15.194694 13695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:15.194646 13692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:15.194736 13693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:15.195576 13687 server_base.cc:1034] running on GCE node
I20260811 03:11:15.196173 13687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:15.197098 13687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:15.198424 13687 hybrid_clock.cc:648] HybridClock initialized: now 1786417875198365 us; error 30 us; skew 500 ppm
I20260811 03:11:15.198787 13687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:15 dist-test-slave-tm1g krb5kdc[13412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417875, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:15.205658 13687 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:11:15.207337 13687 webserver.cc:460] Webserver started at http://127.11.60.2:44935/ using document root <none> and password file <none>
I20260811 03:11:15.208058 13687 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:15.208176 13687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:15.208454 13687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:15.211555 13687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "475f04462fe74bcdbdd8eb1545642544"
format_stamp: "Formatted at 2026-08-11 03:11:15 on dist-test-slave-tm1g"
server_key: "77105c3d7f5f79fc51d4b0c03cff82e3"
server_key_iv: "7eb43301464927d32c7fb412e967f24d"
server_key_version: "encryptionkey@0"
I20260811 03:11:15.212289 13687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "475f04462fe74bcdbdd8eb1545642544"
format_stamp: "Formatted at 2026-08-11 03:11:15 on dist-test-slave-tm1g"
server_key: "77105c3d7f5f79fc51d4b0c03cff82e3"
server_key_iv: "7eb43301464927d32c7fb412e967f24d"
server_key_version: "encryptionkey@0"
I20260811 03:11:15.217702 13687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:11:15.221380 13704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:15.223172 13687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:15.223351 13687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "475f04462fe74bcdbdd8eb1545642544"
format_stamp: "Formatted at 2026-08-11 03:11:15 on dist-test-slave-tm1g"
server_key: "77105c3d7f5f79fc51d4b0c03cff82e3"
server_key_iv: "7eb43301464927d32c7fb412e967f24d"
server_key_version: "encryptionkey@0"
I20260811 03:11:15.223546 13687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:15.237176 13687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:15.241490 13687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:15.241783 13687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:15.244423 13687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:15.244553 13687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:15.244664 13687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:15.244757 13687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 03:11:15 dist-test-slave-tm1g krb5kdc[13412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417875, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:15.269134 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.245254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:51171 (local address 127.11.60.62:46435)
0811 03:11:15.245559 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:15.245566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:15.246812 (+ 1246us) server_negotiation.cc:407] Connection header received
0811 03:11:15.249004 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:15.249010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:15.249110 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:15.249221 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:15.251295 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.252401 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.255131 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.255795 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.255940 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.260872 (+ 4932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:15.260877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:15.260880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:15.260921 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:15.265241 (+ 4320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:15.266045 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:15.266054 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:15.266060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:15.266242 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:15.266927 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:15.266936 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:15.266938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:15.267335 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:15.267500 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:15.268539 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0811 03:11:15.268770 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 03:11:15.270165 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.245714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:46435 (local address 127.11.60.2:51171)
0811 03:11:15.246617 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:15.246670 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:15.248709 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:15.249470 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:15.249479 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:15.250094 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:15.251095 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.251128 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.252557 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.252562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.254934 (+ 2372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.254949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.256063 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.256073 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.256419 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.257228 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:15.257268 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:15.260611 (+ 3343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:15.265448 (+ 4837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:15.265455 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:15.265467 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:15.265903 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:15.266474 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:15.266482 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:15.266487 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:15.266783 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:15.267567 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:15.267573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:15.268088 (+ 515us) client_negotiation.cc:769] Sending connection context
0811 03:11:15.268404 (+ 316us) client_negotiation.cc:240] Negotiation successful
0811 03:11:15.268972 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":362,"threads_started":1}
I20260811 03:11:15.291126 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.276871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:58003 (local address 127.11.60.62:46435)
0811 03:11:15.277127 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:15.277135 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:15.277939 (+ 804us) server_negotiation.cc:407] Connection header received
0811 03:11:15.278256 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:15.278263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:15.278387 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:15.278518 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:15.279933 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.280670 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.282037 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.282704 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.282848 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.284902 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:15.284911 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:15.284916 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:15.284962 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:15.288402 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:15.289086 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:15.289092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:15.289095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:15.289245 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:15.289610 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:15.289615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:15.289618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:15.289944 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:15.290093 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:15.290554 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 03:11:15.290780 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 03:11:15.291275 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.276900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:46435 (local address 127.11.60.2:58003)
0811 03:11:15.277800 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:15.277831 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:15.278042 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:15.278636 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:15.278643 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:15.279431 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:15.279796 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.279807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.280823 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.280829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.281889 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.281900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.282901 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.282908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.283176 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.283988 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:15.284005 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:15.284713 (+ 708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:15.288532 (+ 3819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:15.288537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:15.288541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:15.288954 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:15.289365 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:15.289370 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:15.289373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:15.289501 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:15.290072 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:15.290077 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:15.290259 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 03:11:15.290467 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 03:11:15.290815 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":516,"threads_started":1}
I20260811 03:11:15.303905 13687 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:37977
I20260811 03:11:15.303962 13839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:37977 every 8 connection(s)
I20260811 03:11:15.305195 13687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:15.307350 13840 heartbeater.cc:347] Connected to a master server at 127.11.60.62:46435
I20260811 03:11:15.307564 13840 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:15.308168 13840 heartbeater.cc:511] Master 127.11.60.62:46435 requested a full tablet report, sending...
I20260811 03:11:15.309518 13460 ts_manager.cc:194] Registered new tserver with Master: 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2:37977)
I20260811 03:11:15.310407 13460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:51171
I20260811 03:11:15.314052 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 13687
I20260811 03:11:15.314235 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:15.314775 11504 external_mini_cluster.cc:1428] Setting key 5d3a7617557553d67bfe9aea16d5a8c9
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:11:15.362115 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:46435
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:15.702080 13846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:15.702430 13846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:15.702610 13846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:15.710842 13846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:15.711005 13846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:15.711162 13846 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 03:11:15.711253 13846 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 03:11:15.711393 13846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:15.722071 13846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.60.62:46435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:15.723559 13846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:15.724987 13846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:15.739004 13852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:15.738953 13854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:15.739372 13857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:15.739615 13846 server_base.cc:1034] running on GCE node
I20260811 03:11:15.740551 13846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:15.741940 13846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:15.743212 13846 hybrid_clock.cc:648] HybridClock initialized: now 1786417875743153 us; error 47 us; skew 500 ppm
I20260811 03:11:15.743602 13846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:15 dist-test-slave-tm1g krb5kdc[13412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417875, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:15.747637 13846 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:11:15.749100 13846 webserver.cc:460] Webserver started at http://127.11.60.3:42621/ using document root <none> and password file <none>
I20260811 03:11:15.749763 13846 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:15.749866 13846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:15.750119 13846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:15.753073 13846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "4863db35bebd4c578e6f6c9e043227e3"
format_stamp: "Formatted at 2026-08-11 03:11:15 on dist-test-slave-tm1g"
server_key: "61c816bf917d6b8e1bc0815fee1a9b79"
server_key_iv: "65571bcb12989130ae4051d79543c35d"
server_key_version: "encryptionkey@0"
I20260811 03:11:15.753769 13846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "4863db35bebd4c578e6f6c9e043227e3"
format_stamp: "Formatted at 2026-08-11 03:11:15 on dist-test-slave-tm1g"
server_key: "61c816bf917d6b8e1bc0815fee1a9b79"
server_key_iv: "65571bcb12989130ae4051d79543c35d"
server_key_version: "encryptionkey@0"
I20260811 03:11:15.758960 13846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 03:11:15.762670 13864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:15.764310 13846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:11:15.764554 13846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "4863db35bebd4c578e6f6c9e043227e3"
format_stamp: "Formatted at 2026-08-11 03:11:15 on dist-test-slave-tm1g"
server_key: "61c816bf917d6b8e1bc0815fee1a9b79"
server_key_iv: "65571bcb12989130ae4051d79543c35d"
server_key_version: "encryptionkey@0"
I20260811 03:11:15.764757 13846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:15.776764 13683 heartbeater.cc:503] Master 127.11.60.62:46435 was elected leader, sending a full tablet report...
I20260811 03:11:15.790200 13846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:15.794695 13846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:15.794999 13846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:15.797777 13846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:15.797928 13846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:15.798063 13846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:15.798168 13846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 03:11:15 dist-test-slave-tm1g krb5kdc[13412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417875, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:15.824604 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.798568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:34285 (local address 127.11.60.62:46435)
0811 03:11:15.799164 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:15.799171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:15.800319 (+ 1148us) server_negotiation.cc:407] Connection header received
0811 03:11:15.802890 (+ 2571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:15.802899 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:15.803021 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:15.803110 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:15.805813 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.807023 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.809689 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.810241 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.810419 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.816483 (+ 6064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:15.816493 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:15.816499 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:15.816555 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:15.820225 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:15.821445 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:15.821451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:15.821456 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:15.821607 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:15.822246 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:15.822254 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:15.822258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:15.822743 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:15.822897 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:15.823960 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0811 03:11:15.824191 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":357,"threads_started":1}
I20260811 03:11:15.825729 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.799234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:46435 (local address 127.11.60.3:34285)
0811 03:11:15.800140 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:15.800185 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:15.802477 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:15.803678 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:15.803693 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:15.804562 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:15.805538 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.805571 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.807242 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.807247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.809545 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.809557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.810539 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.810547 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.810959 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.811765 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:15.811814 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:15.816179 (+ 4365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:15.820399 (+ 4220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:15.820412 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:15.820435 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:15.821284 (+ 849us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:15.821750 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:15.821755 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:15.821759 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:15.822132 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:15.822878 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:15.822886 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:15.823491 (+ 605us) client_negotiation.cc:769] Sending connection context
0811 03:11:15.823881 (+ 390us) client_negotiation.cc:240] Negotiation successful
0811 03:11:15.824440 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":395,"threads_started":1}
I20260811 03:11:15.848554 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.832518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:40519 (local address 127.11.60.62:46435)
0811 03:11:15.832681 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:15.832685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:15.833490 (+ 805us) server_negotiation.cc:407] Connection header received
0811 03:11:15.833683 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:15.833687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:15.833756 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:15.833859 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:15.835127 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.836602 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.838081 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.838952 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.839103 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.841437 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:15.841446 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:15.841453 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:15.841505 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:15.845223 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:15.846175 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:15.846182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:15.846186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:15.846393 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:15.846931 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:15.846936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:15.846939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:15.847264 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:15.847386 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:15.848049 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 03:11:15.848293 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 03:11:15.849076 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.832620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:46435 (local address 127.11.60.3:40519)
0811 03:11:15.833368 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:15.833384 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:15.833540 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:15.833884 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:15.833889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:15.834435 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:15.834904 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.834930 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.836790 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.836794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.837939 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.837952 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.839201 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.839209 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.839384 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.840149 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:15.840168 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:15.841230 (+ 1062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:15.845378 (+ 4148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:15.845384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:15.845388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:15.846038 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:15.846511 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:15.846516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:15.846519 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:15.846722 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:15.847426 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:15.847432 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:15.847696 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 03:11:15.847910 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 03:11:15.848391 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":393,"threads_started":1}
I20260811 03:11:15.853751 13846 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:35873
I20260811 03:11:15.853856 13997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:35873 every 8 connection(s)
I20260811 03:11:15.855116 13846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:15.857254 13999 heartbeater.cc:347] Connected to a master server at 127.11.60.62:46435
I20260811 03:11:15.857511 13999 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:15.858157 13999 heartbeater.cc:511] Master 127.11.60.62:46435 requested a full tablet report, sending...
I20260811 03:11:15.859597 13460 ts_manager.cc:194] Registered new tserver with Master: 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873)
I20260811 03:11:15.860463 13460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:34285
I20260811 03:11:15.864114 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 13846
I20260811 03:11:15.864269 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:15.864658 11504 external_mini_cluster.cc:1428] Setting key 4be23c95bb5741a431eaab75c430b153
I20260811 03:11:15.867729 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 03:11:15 dist-test-slave-tm1g krb5kdc[13412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 03:11:15 dist-test-slave-tm1g krb5kdc[13412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:15.923504 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.909413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.11.60.62:46435)
0811 03:11:15.909667 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:15.909674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:15.909965 (+ 291us) server_negotiation.cc:407] Connection header received
0811 03:11:15.910179 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:15.910184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:15.910269 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:15.910408 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:15.911977 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.912784 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.914003 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.914464 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.914559 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.918487 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:15.918492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:15.918495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:15.918530 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:15.921010 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:15.921588 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:15.921592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:15.921594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:15.921727 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:15.922058 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:15.922061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:15.922063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:15.922460 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:15.922654 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:15.922946 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 03:11:15.923108 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 03:11:15.928309 13460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:11:15.935977 13460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:15.976065 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.950670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34074 (local address 127.11.60.2:37977)
0811 03:11:15.965681 (+ 15011us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:15.965695 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:15.965736 (+ 41us) server_negotiation.cc:407] Connection header received
0811 03:11:15.965852 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:15.965861 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:15.966184 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:15.966448 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:15.967567 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.969579 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.973370 (+ 3791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.974435 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.974616 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.974822 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:15.975553 (+ 731us) server_negotiation.cc:299] Negotiation successful
0811 03:11:15.975751 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14780,"thread_start_us":14621,"threads_started":1}
I20260811 03:11:15.976516 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.950042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:37977 (local address 127.0.0.1:34074)
0811 03:11:15.964596 (+ 14554us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:15.964630 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:15.964793 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:15.966722 (+ 1929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:15.966731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:15.966762 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:15.967290 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.967308 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.970291 (+ 2983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.970330 (+ 39us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.973206 (+ 2876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.973218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.974919 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.974925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.975210 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.975320 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 03:11:15.975494 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 03:11:15.975732 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14369,"mutex_wait_us":37,"thread_start_us":19872,"threads_started":1}
I20260811 03:11:15.980091 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.951840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60454 (local address 127.11.60.1:37523)
0811 03:11:15.971075 (+ 19235us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:15.971085 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:15.971120 (+ 35us) server_negotiation.cc:407] Connection header received
0811 03:11:15.971236 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:15.971246 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:15.971604 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:15.972787 (+ 1183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:15.973026 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.974653 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.977397 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.978934 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.979076 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.979275 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:15.979637 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 03:11:15.979802 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19052,"thread_start_us":18903,"threads_started":1}
I20260811 03:11:15.980252 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.951539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:37523 (local address 127.0.0.1:60454)
0811 03:11:15.970269 (+ 18730us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:15.970295 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:15.970458 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:15.971917 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:15.971923 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:15.971950 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:15.972542 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.972562 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.974794 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.974801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.977202 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.977218 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.979184 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.979191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.979415 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.979456 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 03:11:15.979564 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 03:11:15.979706 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18537,"mutex_wait_us":41,"thread_start_us":13384,"threads_started":1}
I20260811 03:11:15.980892 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.949544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:35873 (local address 127.0.0.1:48156)
0811 03:11:15.951097 (+ 1553us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:15.951135 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:15.951299 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:15.972369 (+ 21070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:15.972377 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:15.972407 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:15.972919 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.972932 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.975277 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.975281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.977519 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:15.977532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.979973 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.979980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:15.980461 (+ 481us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.980503 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 03:11:15.980565 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 03:11:15.980671 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1323,"thread_start_us":1184,"threads_started":1}
I20260811 03:11:15.980907 13761 tablet_service.cc:1467] Processing CreateTablet for tablet 991db90cfead464ea5cd5c3bf0e4488b (DEFAULT_TABLE table=test-table [id=64e742f4234447fa9c107a965a113086]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:15.982188 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:15.949936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.11.60.3:35873)
0811 03:11:15.971630 (+ 21694us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:15.971641 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:15.971680 (+ 39us) server_negotiation.cc:407] Connection header received
0811 03:11:15.971814 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:15.971823 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:15.972088 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:15.973129 (+ 1041us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:15.973505 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.975128 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.978407 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:15.979730 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:15.981258 (+ 1528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:15.981588 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:15.981682 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 03:11:15.981845 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21399,"thread_start_us":21225,"threads_started":1}
I20260811 03:11:15.982489 13761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 991db90cfead464ea5cd5c3bf0e4488b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:15.984256 13606 tablet_service.cc:1467] Processing CreateTablet for tablet 991db90cfead464ea5cd5c3bf0e4488b (DEFAULT_TABLE table=test-table [id=64e742f4234447fa9c107a965a113086]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:15.985792 13606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 991db90cfead464ea5cd5c3bf0e4488b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:15.986121 13921 tablet_service.cc:1467] Processing CreateTablet for tablet 991db90cfead464ea5cd5c3bf0e4488b (DEFAULT_TABLE table=test-table [id=64e742f4234447fa9c107a965a113086]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:15.987599 13921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 991db90cfead464ea5cd5c3bf0e4488b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:15.994989 14019 tablet_bootstrap.cc:492] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: Bootstrap starting.
I20260811 03:11:15.998256 14019 tablet_bootstrap.cc:654] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:15.999380 14019 log.cc:824] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:16.001842 14021 tablet_bootstrap.cc:492] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: Bootstrap starting.
I20260811 03:11:16.002663 14019 tablet_bootstrap.cc:492] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: No bootstrap required, opened a new log
I20260811 03:11:16.002960 14019 ts_tablet_manager.cc:1397] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260811 03:11:16.005564 14021 tablet_bootstrap.cc:654] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:16.006881 14021 log.cc:824] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:16.007228 14022 tablet_bootstrap.cc:492] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: Bootstrap starting.
I20260811 03:11:16.010226 14021 tablet_bootstrap.cc:492] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: No bootstrap required, opened a new log
I20260811 03:11:16.010550 14021 ts_tablet_manager.cc:1397] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 03:11:16.011161 14022 tablet_bootstrap.cc:654] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:16.012408 14022 log.cc:824] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:16.012816 14019 raft_consensus.cc:348] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } }
I20260811 03:11:16.013291 14019 raft_consensus.cc:374] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:16.013415 14019 raft_consensus.cc:729] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475f04462fe74bcdbdd8eb1545642544, State: Initialized, Role: FOLLOWER
I20260811 03:11:16.014092 14019 consensus_queue.cc:260] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } }
I20260811 03:11:16.015270 13840 heartbeater.cc:503] Master 127.11.60.62:46435 was elected leader, sending a full tablet report...
I20260811 03:11:16.015689 14022 tablet_bootstrap.cc:492] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: No bootstrap required, opened a new log
I20260811 03:11:16.015957 14022 ts_tablet_manager.cc:1397] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 03:11:16.019361 14021 raft_consensus.cc:348] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } }
I20260811 03:11:16.019814 14021 raft_consensus.cc:374] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:16.019856 14019 ts_tablet_manager.cc:1428] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: Time spent starting tablet: real 0.017s user 0.012s sys 0.000s
I20260811 03:11:16.019917 14021 raft_consensus.cc:729] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f699cf80b4f94fb0baae8cbf8e308269, State: Initialized, Role: FOLLOWER
I20260811 03:11:16.020521 14021 consensus_queue.cc:260] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } }
I20260811 03:11:16.022490 14021 ts_tablet_manager.cc:1428] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 03:11:16.027371 14022 raft_consensus.cc:348] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } }
I20260811 03:11:16.027819 14022 raft_consensus.cc:374] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:16.027925 14022 raft_consensus.cc:729] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4863db35bebd4c578e6f6c9e043227e3, State: Initialized, Role: FOLLOWER
I20260811 03:11:16.028575 14022 consensus_queue.cc:260] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } }
I20260811 03:11:16.029785 13999 heartbeater.cc:503] Master 127.11.60.62:46435 was elected leader, sending a full tablet report...
I20260811 03:11:16.030401 14022 ts_tablet_manager.cc:1428] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260811 03:11:16.033836 14027 raft_consensus.cc:482] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:16.034195 14027 raft_consensus.cc:504] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } }
I20260811 03:11:16.035866 14027 leader_election.cc:288] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873), 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2:37977)
I20260811 03:11:16.046200 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.036378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:41115 (local address 127.11.60.2:37977)
0811 03:11:16.036621 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.036630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.037806 (+ 1176us) server_negotiation.cc:407] Connection header received
0811 03:11:16.038094 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.038100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.038198 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.038362 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:16.039146 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.040329 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.043515 (+ 3186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.044740 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.044884 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.044996 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.045784 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.045917 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 03:11:16.046190 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.036319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:51937 (local address 127.11.60.3:35873)
0811 03:11:16.036525 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.036529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.037884 (+ 1355us) server_negotiation.cc:407] Connection header received
0811 03:11:16.038340 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.038347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.038450 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.038585 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:16.039308 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.040360 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.043499 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.044737 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.044884 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.045023 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.045784 (+ 761us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.045917 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 03:11:16.046474 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.036416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:35873 (local address 127.11.60.1:51937)
0811 03:11:16.037705 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:16.037721 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:16.038007 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:16.038601 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:16.038606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:16.038626 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:16.039170 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.039180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.040503 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.040507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.043329 (+ 2822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.043345 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.045094 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.045101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.045521 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.045634 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 03:11:16.045744 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 03:11:16.045969 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1145,"mutex_wait_us":475,"thread_start_us":481,"threads_started":1}
I20260811 03:11:16.046818 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.036416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:37977 (local address 127.11.60.1:41115)
0811 03:11:16.037637 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:16.037667 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:16.037843 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:16.038372 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:16.038376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:16.038402 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:16.038947 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.038966 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.040502 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.040507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.043329 (+ 2822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.043345 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.045097 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.045102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.045525 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.045633 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 03:11:16.045744 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 03:11:16.045969 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":384,"thread_start_us":457,"threads_started":1}
I20260811 03:11:16.047617 13942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991db90cfead464ea5cd5c3bf0e4488b" candidate_uuid: "f699cf80b4f94fb0baae8cbf8e308269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4863db35bebd4c578e6f6c9e043227e3" is_pre_election: true
I20260811 03:11:16.047758 13790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991db90cfead464ea5cd5c3bf0e4488b" candidate_uuid: "f699cf80b4f94fb0baae8cbf8e308269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "475f04462fe74bcdbdd8eb1545642544" is_pre_election: true
I20260811 03:11:16.048218 13942 raft_consensus.cc:2454] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f699cf80b4f94fb0baae8cbf8e308269 in term 0.
I20260811 03:11:16.048275 13790 raft_consensus.cc:2454] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f699cf80b4f94fb0baae8cbf8e308269 in term 0.
I20260811 03:11:16.048944 13555 leader_election.cc:302] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4863db35bebd4c578e6f6c9e043227e3, f699cf80b4f94fb0baae8cbf8e308269; no voters:
I20260811 03:11:16.049413 14027 raft_consensus.cc:2781] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:16.049576 14027 raft_consensus.cc:482] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:16.049683 14027 raft_consensus.cc:3037] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:16.051966 14027 raft_consensus.cc:504] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } }
I20260811 03:11:16.052655 14027 leader_election.cc:288] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [CANDIDATE]: Term 1 election: Requested vote from peers 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873), 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2:37977)
I20260811 03:11:16.053027 13942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991db90cfead464ea5cd5c3bf0e4488b" candidate_uuid: "f699cf80b4f94fb0baae8cbf8e308269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4863db35bebd4c578e6f6c9e043227e3"
I20260811 03:11:16.053244 13942 raft_consensus.cc:3037] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:16.053179 13790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991db90cfead464ea5cd5c3bf0e4488b" candidate_uuid: "f699cf80b4f94fb0baae8cbf8e308269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "475f04462fe74bcdbdd8eb1545642544"
I20260811 03:11:16.053422 13790 raft_consensus.cc:3037] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:16.055649 13942 raft_consensus.cc:2454] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f699cf80b4f94fb0baae8cbf8e308269 in term 1.
I20260811 03:11:16.055799 13790 raft_consensus.cc:2454] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f699cf80b4f94fb0baae8cbf8e308269 in term 1.
I20260811 03:11:16.056092 13555 leader_election.cc:302] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4863db35bebd4c578e6f6c9e043227e3, f699cf80b4f94fb0baae8cbf8e308269; no voters:
I20260811 03:11:16.056422 14027 raft_consensus.cc:2781] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:16.056934 14027 raft_consensus.cc:686] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 1 LEADER]: Becoming Leader. State: Replica: f699cf80b4f94fb0baae8cbf8e308269, State: Running, Role: LEADER
W20260811 03:11:16.057288 13841 tablet.cc:2369] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:16.057507 14027 consensus_queue.cc:237] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } }
I20260811 03:11:16.062675 13460 catalog_manager.cc:5495] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 reported cstate change: term changed from 0 to 1, leader changed from <none> to f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1). New cstate: current_term: 1 leader_uuid: "f699cf80b4f94fb0baae8cbf8e308269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } health_report { overall_health: UNKNOWN } } }
W20260811 03:11:16.106931 14000 tablet.cc:2369] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 03:11:16 dist-test-slave-tm1g krb5kdc[13412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:16.123415 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.109242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.11.60.62:46435)
0811 03:11:16.109464 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.109472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.110137 (+ 665us) server_negotiation.cc:407] Connection header received
0811 03:11:16.110274 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.110278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.110395 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.110514 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:16.111717 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.112505 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.113764 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.114199 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.114382 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.117812 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:16.117817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:16.117820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:16.117851 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:16.120473 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:16.121382 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:16.121386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:16.121388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:16.121526 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:16.121907 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:16.121910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:16.121912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:16.122182 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:16.122290 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.122807 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.123122 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":86}
I20260811 03:11:16.123454 14041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.109383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:46435 (local address 127.0.0.1:35708)
0811 03:11:16.110025 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:16.110041 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:16.110169 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:16.110573 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:16.110578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:16.111235 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:16.111544 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.111554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.112644 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.112648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.113624 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.113634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.114454 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.114459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.114582 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.115264 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:16.115282 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:16.117613 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:16.120606 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:16.120619 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:16.120627 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:16.121250 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:16.121627 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:16.121632 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:16.121636 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:16.121806 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:16.122298 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:16.122339 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:16.122533 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 03:11:16.122711 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 03:11:16.122991 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":371,"threads_started":1}
I20260811 03:11:16.129436 13460 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:35708:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 03:11:16.130574 13460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:16.137763 13606 tablet_service.cc:1467] Processing CreateTablet for tablet 0567c525f5a04519b518c18a05f92f10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbbb770ee9184539a9510c83b081012c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 03:11:16.138127 13921 tablet_service.cc:1467] Processing CreateTablet for tablet 0567c525f5a04519b518c18a05f92f10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbbb770ee9184539a9510c83b081012c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 03:11:16.138834 13606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0567c525f5a04519b518c18a05f92f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:16.138518 13761 tablet_service.cc:1467] Processing CreateTablet for tablet 0567c525f5a04519b518c18a05f92f10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbbb770ee9184539a9510c83b081012c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 03:11:16.139070 13921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0567c525f5a04519b518c18a05f92f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:16.139297 13761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0567c525f5a04519b518c18a05f92f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:16.144539 14021 tablet_bootstrap.cc:492] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: Bootstrap starting.
I20260811 03:11:16.145906 14021 tablet_bootstrap.cc:654] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:16.146385 14022 tablet_bootstrap.cc:492] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: Bootstrap starting.
I20260811 03:11:16.146477 14019 tablet_bootstrap.cc:492] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: Bootstrap starting.
I20260811 03:11:16.148164 14022 tablet_bootstrap.cc:654] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:16.148216 14019 tablet_bootstrap.cc:654] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:16.148620 14021 tablet_bootstrap.cc:492] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: No bootstrap required, opened a new log
I20260811 03:11:16.148778 14021 ts_tablet_manager.cc:1397] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 03:11:16.149749 14021 raft_consensus.cc:348] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.150014 14021 raft_consensus.cc:374] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:16.150118 14021 raft_consensus.cc:729] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f699cf80b4f94fb0baae8cbf8e308269, State: Initialized, Role: FOLLOWER
I20260811 03:11:16.150413 14021 consensus_queue.cc:260] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.150672 14022 tablet_bootstrap.cc:492] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: No bootstrap required, opened a new log
I20260811 03:11:16.151139 14021 ts_tablet_manager.cc:1428] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 03:11:16.150980 14027 tablet_replica.cc:439] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } }
I20260811 03:11:16.151371 14027 tablet_replica.cc:442] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:16.151513 14022 ts_tablet_manager.cc:1397] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260811 03:11:16.152182 14022 raft_consensus.cc:348] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.152411 14022 raft_consensus.cc:374] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:16.152508 14022 raft_consensus.cc:729] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4863db35bebd4c578e6f6c9e043227e3, State: Initialized, Role: FOLLOWER
I20260811 03:11:16.152760 14022 consensus_queue.cc:260] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.153415 14022 ts_tablet_manager.cc:1428] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 03:11:16.153254 14028 tablet_replica.cc:439] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } }
I20260811 03:11:16.153641 14028 tablet_replica.cc:442] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:16.156855 14019 tablet_bootstrap.cc:492] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: No bootstrap required, opened a new log
I20260811 03:11:16.157032 14019 ts_tablet_manager.cc:1397] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260811 03:11:16.157662 14019 raft_consensus.cc:348] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.157888 14019 raft_consensus.cc:374] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:16.157986 14019 raft_consensus.cc:729] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475f04462fe74bcdbdd8eb1545642544, State: Initialized, Role: FOLLOWER
I20260811 03:11:16.158214 14019 consensus_queue.cc:260] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.158960 14019 ts_tablet_manager.cc:1428] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 03:11:16.158757 14026 tablet_replica.cc:439] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } }
I20260811 03:11:16.159179 14026 tablet_replica.cc:442] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:16.182729 14026 raft_consensus.cc:482] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:16.183018 14026 raft_consensus.cc:504] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.184644 14026 leader_election.cc:288] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523), 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873)
I20260811 03:11:16.193598 14046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.185092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:37523 (local address 127.11.60.2:40303)
0811 03:11:16.185895 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:16.185917 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:16.186150 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:16.186796 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:16.186799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:16.186821 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:16.187297 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.187311 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.188880 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.188926 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.191366 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.191380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.192969 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.192974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.193102 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.193128 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 03:11:16.193215 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 03:11:16.193309 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":484,"threads_started":1}
I20260811 03:11:16.193914 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.185106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:40303 (local address 127.11.60.1:37523)
0811 03:11:16.185300 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.185304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.186013 (+ 709us) server_negotiation.cc:407] Connection header received
0811 03:11:16.186379 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.186385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.186504 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.186580 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:16.187477 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.188731 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.191549 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.192704 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.192852 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.192957 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.193267 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.193416 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 03:11:16.194568 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.186340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:44459 (local address 127.11.60.3:35873)
0811 03:11:16.186632 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.186638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.187014 (+ 376us) server_negotiation.cc:407] Connection header received
0811 03:11:16.187291 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.187297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.187385 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.187500 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:16.188156 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.189424 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.192349 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.193592 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.193737 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.193828 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.194207 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.194349 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260811 03:11:16.194708 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.186030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:35873 (local address 127.11.60.2:44459)
0811 03:11:16.186892 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:16.186916 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:16.187065 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:16.187509 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:16.187514 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:16.187572 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:16.187962 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.187974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.189592 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.189598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.192096 (+ 2498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.192132 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.193848 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.193858 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.194040 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.194079 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 03:11:16.194181 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 03:11:16.194378 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":557,"threads_started":1}
I20260811 03:11:16.194908 13630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0567c525f5a04519b518c18a05f92f10" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f699cf80b4f94fb0baae8cbf8e308269" is_pre_election: true
I20260811 03:11:16.195516 13942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0567c525f5a04519b518c18a05f92f10" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4863db35bebd4c578e6f6c9e043227e3" is_pre_election: true
I20260811 03:11:16.195539 13630 raft_consensus.cc:2454] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 0.
I20260811 03:11:16.195806 13942 raft_consensus.cc:2454] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 0.
I20260811 03:11:16.196219 13709 leader_election.cc:302] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475f04462fe74bcdbdd8eb1545642544, f699cf80b4f94fb0baae8cbf8e308269; no voters:
I20260811 03:11:16.196641 14026 raft_consensus.cc:2781] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:16.196782 14026 raft_consensus.cc:482] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:16.196913 14026 raft_consensus.cc:3037] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:16.199193 14026 raft_consensus.cc:504] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.199770 14026 leader_election.cc:288] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 election: Requested vote from peers f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523), 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873)
I20260811 03:11:16.200330 13630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0567c525f5a04519b518c18a05f92f10" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f699cf80b4f94fb0baae8cbf8e308269"
I20260811 03:11:16.200448 13942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0567c525f5a04519b518c18a05f92f10" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4863db35bebd4c578e6f6c9e043227e3"
I20260811 03:11:16.200539 13630 raft_consensus.cc:3037] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:16.200716 13942 raft_consensus.cc:3037] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:16.202936 13630 raft_consensus.cc:2454] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 1.
I20260811 03:11:16.202973 13942 raft_consensus.cc:2454] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 1.
I20260811 03:11:16.203344 13709 leader_election.cc:302] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475f04462fe74bcdbdd8eb1545642544, f699cf80b4f94fb0baae8cbf8e308269; no voters:
I20260811 03:11:16.203689 14026 raft_consensus.cc:2781] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:16.204276 14026 raft_consensus.cc:686] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [term 1 LEADER]: Becoming Leader. State: Replica: 475f04462fe74bcdbdd8eb1545642544, State: Running, Role: LEADER
I20260811 03:11:16.204952 14026 consensus_queue.cc:237] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.208222 14048 tablet_replica.cc:439] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: TxnStatusTablet state changed. Reason: New leader 475f04462fe74bcdbdd8eb1545642544. Latest consensus state: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } }
I20260811 03:11:16.208609 14048 tablet_replica.cc:442] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: This TxnStatusTablet replica's current role is: LEADER
I20260811 03:11:16.209792 14051 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 03:11:16.210079 14051 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 03:11:16.210556 13460 catalog_manager.cc:5495] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 reported cstate change: term changed from 0 to 1, leader changed from <none> to 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2). New cstate: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } health_report { overall_health: UNKNOWN } } }
I20260811 03:11:16.241968 14041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.235382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:37977 (local address 127.0.0.1:34080)
0811 03:11:16.235621 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:16.235642 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:16.235773 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:16.236288 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:16.236292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:16.236302 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:11:16.236492 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.236500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.238006 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.238009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.239186 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.239198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.240200 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.240205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.240377 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.240433 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:11:16.241500 (+ 1067us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:11:16.241520 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 03:11:16.241592 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 03:11:16.241711 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 03:11:16.242053 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.235424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34080 (local address 127.11.60.2:37977)
0811 03:11:16.235621 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.235627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.235725 (+ 98us) server_negotiation.cc:407] Connection header received
0811 03:11:16.235965 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.235971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.236088 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.236261 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:16.236619 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.237850 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.239332 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.240020 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.240153 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.240626 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:16.241365 (+ 739us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:16.241525 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.241636 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.241724 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:11:16.251019 13630 raft_consensus.cc:1261] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Refusing update from remote peer 475f04462fe74bcdbdd8eb1545642544: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:16.251389 13942 raft_consensus.cc:1261] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Refusing update from remote peer 475f04462fe74bcdbdd8eb1545642544: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:16.252116 14048 consensus_queue.cc:1035] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:16.252493 14026 consensus_queue.cc:1035] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:16.264438 14028 tablet_replica.cc:439] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: TxnStatusTablet state changed. Reason: New leader 475f04462fe74bcdbdd8eb1545642544. Latest consensus state: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } }
I20260811 03:11:16.264680 14028 tablet_replica.cc:442] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 03:11:16.269639 13684 tablet.cc:2369] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:16.273995 14028 tablet_replica.cc:439] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } }
I20260811 03:11:16.274276 14028 tablet_replica.cc:442] T 0567c525f5a04519b518c18a05f92f10 P 4863db35bebd4c578e6f6c9e043227e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:16.279012 14026 tablet_replica.cc:439] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } }
I20260811 03:11:16.279296 14026 tablet_replica.cc:442] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: This TxnStatusTablet replica's current role is: LEADER
I20260811 03:11:16.279794 14027 tablet_replica.cc:439] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: TxnStatusTablet state changed. Reason: New leader 475f04462fe74bcdbdd8eb1545642544. Latest consensus state: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } }
I20260811 03:11:16.280158 14027 tablet_replica.cc:442] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:16.284058 14048 tablet_replica.cc:439] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } }
I20260811 03:11:16.284328 14048 tablet_replica.cc:442] T 0567c525f5a04519b518c18a05f92f10 P 475f04462fe74bcdbdd8eb1545642544: This TxnStatusTablet replica's current role is: LEADER
I20260811 03:11:16.290293 14027 tablet_replica.cc:439] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } } }
I20260811 03:11:16.290649 14027 tablet_replica.cc:442] T 0567c525f5a04519b518c18a05f92f10 P f699cf80b4f94fb0baae8cbf8e308269: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:11:16.304126 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.297262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60462 (local address 127.11.60.1:37523)
0811 03:11:16.297827 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.297831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.297845 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:11:16.297922 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.297926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.298026 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.298145 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:16.298645 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.299513 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.301618 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.302023 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.302133 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.302763 (+ 630us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:16.303408 (+ 645us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:16.303561 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.303681 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.303767 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466}
I20260811 03:11:16.321122 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.315774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:36971 (local address 127.11.60.2:37977)
0811 03:11:16.315968 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.315975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.315998 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:11:16.316263 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.316267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.316335 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.316459 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:16.316972 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.317784 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.319279 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.319745 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.319897 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.320379 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:16.320482 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:16.320584 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.320725 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.320801 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:11:16.321225 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.315249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:37977 (local address 127.11.60.1:36971)
0811 03:11:16.315897 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:16.315939 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:16.316084 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:16.316474 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:16.316479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:16.316491 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:11:16.316813 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.316822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.317920 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.317926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.319130 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.319143 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.319939 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.319948 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.320076 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.320133 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:11:16.320584 (+ 451us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:11:16.320597 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 03:11:16.320701 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 03:11:16.320837 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":601,"threads_started":1}
I20260811 03:11:16.335273 13790 raft_consensus.cc:1261] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Refusing update from remote peer f699cf80b4f94fb0baae8cbf8e308269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:16.336472 14027 consensus_queue.cc:1035] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:16.338187 13941 raft_consensus.cc:1261] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Refusing update from remote peer f699cf80b4f94fb0baae8cbf8e308269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:16.339892 14027 consensus_queue.cc:1035] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:16.344461 14055 mvcc.cc:205] Tried to move back new op lower bound from 7317167621460008960 to 7317167620336173056. Current Snapshot: MvccSnapshot[applied={T|T < 7317167621460008960}]
I20260811 03:11:16.346843 14056 mvcc.cc:205] Tried to move back new op lower bound from 7317167621460008960 to 7317167620336173056. Current Snapshot: MvccSnapshot[applied={T|T < 7317167621460008960}]
I20260811 03:11:16.347571 14057 mvcc.cc:205] Tried to move back new op lower bound from 7317167621460008960 to 7317167620336173056. Current Snapshot: MvccSnapshot[applied={T|T < 7317167621460008960}]
I20260811 03:11:16.384845 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.372479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:36271 (local address 127.11.60.1:37523)
0811 03:11:16.372981 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.372990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.377669 (+ 4679us) server_negotiation.cc:407] Connection header received
0811 03:11:16.377893 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.377899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.377992 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.378062 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:16.378891 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.379798 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.382081 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.382874 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.383017 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.383702 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:16.383882 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:16.384008 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.384313 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.384409 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303}
I20260811 03:11:16.384895 14074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.371742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:37523 (local address 127.11.60.2:36271)
0811 03:11:16.377536 (+ 5794us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:16.377558 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:16.377717 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:16.378277 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:16.378282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:16.378295 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:11:16.378719 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.378738 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.379935 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.379941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.381831 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.381852 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.383090 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.383097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.383318 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.383423 (+ 105us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:11:16.384010 (+ 587us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:11:16.384030 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 03:11:16.384265 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 03:11:16.384443 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5576,"thread_start_us":5436,"threads_started":1}
I20260811 03:11:16.427686 13458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:16.428085 13458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:16.431972 13458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df: Sending DeleteTablet for 3 replicas of tablet 991db90cfead464ea5cd5c3bf0e4488b
I20260811 03:11:16.433255 13921 tablet_service.cc:1514] Processing DeleteTablet for tablet 991db90cfead464ea5cd5c3bf0e4488b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:16 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:48156
I20260811 03:11:16.434132 14078 tablet_replica.cc:330] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: stopping tablet replica
I20260811 03:11:16.434047 13761 tablet_service.cc:1514] Processing DeleteTablet for tablet 991db90cfead464ea5cd5c3bf0e4488b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:16 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:34074
I20260811 03:11:16.434799 13606 tablet_service.cc:1514] Processing DeleteTablet for tablet 991db90cfead464ea5cd5c3bf0e4488b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:16 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:60454
I20260811 03:11:16.434864 14078 raft_consensus.cc:2229] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:16.435271 14078 raft_consensus.cc:2258] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:16.437588 14078 ts_tablet_manager.cc:1905] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:16.442350 13460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:11:16.443064 13460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:16.453735 13760 tablet_service.cc:1467] Processing CreateTablet for tablet 55c02e56d74540558c8d71235d754cd5 (DEFAULT_TABLE table=test-table [id=0dfec52230de405cb493540b07fead01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:16.454401 13760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55c02e56d74540558c8d71235d754cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:16.454262 13605 tablet_service.cc:1467] Processing CreateTablet for tablet 55c02e56d74540558c8d71235d754cd5 (DEFAULT_TABLE table=test-table [id=0dfec52230de405cb493540b07fead01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:16.454919 13605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55c02e56d74540558c8d71235d754cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:16.458956 13921 tablet_service.cc:1467] Processing CreateTablet for tablet 55c02e56d74540558c8d71235d754cd5 (DEFAULT_TABLE table=test-table [id=0dfec52230de405cb493540b07fead01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:16.459445 13921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55c02e56d74540558c8d71235d754cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:16.462059 14019 tablet_bootstrap.cc:492] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544: Bootstrap starting.
I20260811 03:11:16.462895 14081 tablet_replica.cc:330] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: stopping tablet replica
I20260811 03:11:16.463658 14079 tablet_replica.cc:330] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: stopping tablet replica
I20260811 03:11:16.463887 14019 tablet_bootstrap.cc:654] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:16.464131 14079 raft_consensus.cc:2229] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:16.464978 14081 raft_consensus.cc:2229] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:11:16.465709 14081 raft_consensus.cc:2258] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:16.469887 14021 tablet_bootstrap.cc:492] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269: Bootstrap starting.
I20260811 03:11:16.471701 14021 tablet_bootstrap.cc:654] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:16.472605 14022 tablet_bootstrap.cc:492] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3: Bootstrap starting.
I20260811 03:11:16.474606 14079 raft_consensus.cc:2258] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:16.474843 14022 tablet_bootstrap.cc:654] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:16.475595 14019 tablet_bootstrap.cc:492] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544: No bootstrap required, opened a new log
I20260811 03:11:16.475756 14019 ts_tablet_manager.cc:1397] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.003s
I20260811 03:11:16.476585 14019 raft_consensus.cc:348] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.476861 14019 raft_consensus.cc:374] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:16.476948 14019 raft_consensus.cc:729] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475f04462fe74bcdbdd8eb1545642544, State: Initialized, Role: FOLLOWER
I20260811 03:11:16.476979 14078 ts_tablet_manager.cc:1918] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:11:16.477151 14078 log.cc:1196] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/991db90cfead464ea5cd5c3bf0e4488b
I20260811 03:11:16.477245 14019 consensus_queue.cc:260] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.477563 14079 ts_tablet_manager.cc:1905] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:16.477779 14078 ts_tablet_manager.cc:1939] T 991db90cfead464ea5cd5c3bf0e4488b P 4863db35bebd4c578e6f6c9e043227e3: Deleting consensus metadata
I20260811 03:11:16.477835 14019 ts_tablet_manager.cc:1428] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260811 03:11:16.481082 14081 ts_tablet_manager.cc:1905] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:16.485103 14079 ts_tablet_manager.cc:1918] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:11:16.485280 14079 log.cc:1196] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/991db90cfead464ea5cd5c3bf0e4488b
I20260811 03:11:16.485605 13444 catalog_manager.cc:4841] TS 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873): tablet 991db90cfead464ea5cd5c3bf0e4488b (table test-table [id=64e742f4234447fa9c107a965a113086]) successfully deleted
I20260811 03:11:16.485843 14079 ts_tablet_manager.cc:1939] T 991db90cfead464ea5cd5c3bf0e4488b P 475f04462fe74bcdbdd8eb1545642544: Deleting consensus metadata
I20260811 03:11:16.487484 13442 catalog_manager.cc:4841] TS 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2:37977): tablet 991db90cfead464ea5cd5c3bf0e4488b (table test-table [id=64e742f4234447fa9c107a965a113086]) successfully deleted
I20260811 03:11:16.495275 14022 tablet_bootstrap.cc:492] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3: No bootstrap required, opened a new log
I20260811 03:11:16.495566 14022 ts_tablet_manager.cc:1397] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.001s
I20260811 03:11:16.496466 14022 raft_consensus.cc:348] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.496800 14022 raft_consensus.cc:374] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:16.496767 14081 ts_tablet_manager.cc:1918] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:11:16.496969 14081 log.cc:1196] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/991db90cfead464ea5cd5c3bf0e4488b
I20260811 03:11:16.496973 14022 raft_consensus.cc:729] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4863db35bebd4c578e6f6c9e043227e3, State: Initialized, Role: FOLLOWER
I20260811 03:11:16.497422 14022 consensus_queue.cc:260] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.497603 14081 ts_tablet_manager.cc:1939] T 991db90cfead464ea5cd5c3bf0e4488b P f699cf80b4f94fb0baae8cbf8e308269: Deleting consensus metadata
I20260811 03:11:16.499176 14022 ts_tablet_manager.cc:1428] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 03:11:16.500954 14021 tablet_bootstrap.cc:492] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269: No bootstrap required, opened a new log
I20260811 03:11:16.501211 14021 ts_tablet_manager.cc:1397] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269: Time spent bootstrapping tablet: real 0.031s user 0.009s sys 0.003s
I20260811 03:11:16.502123 14021 raft_consensus.cc:348] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.502475 14021 raft_consensus.cc:374] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:16.502630 14021 raft_consensus.cc:729] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f699cf80b4f94fb0baae8cbf8e308269, State: Initialized, Role: FOLLOWER
I20260811 03:11:16.503108 14021 consensus_queue.cc:260] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.504719 13444 catalog_manager.cc:4841] TS f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523): tablet 991db90cfead464ea5cd5c3bf0e4488b (table test-table [id=64e742f4234447fa9c107a965a113086]) successfully deleted
I20260811 03:11:16.506349 14021 ts_tablet_manager.cc:1428] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 03:11:16.686396 14026 raft_consensus.cc:482] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:16.686647 14026 raft_consensus.cc:504] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.687598 14026 leader_election.cc:288] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523), 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873)
I20260811 03:11:16.688295 13630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c02e56d74540558c8d71235d754cd5" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f699cf80b4f94fb0baae8cbf8e308269" is_pre_election: true
I20260811 03:11:16.688318 13941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c02e56d74540558c8d71235d754cd5" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4863db35bebd4c578e6f6c9e043227e3" is_pre_election: true
I20260811 03:11:16.688588 13630 raft_consensus.cc:2454] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 0.
I20260811 03:11:16.688642 13941 raft_consensus.cc:2454] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 0.
I20260811 03:11:16.689146 13709 leader_election.cc:302] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475f04462fe74bcdbdd8eb1545642544, f699cf80b4f94fb0baae8cbf8e308269; no voters:
I20260811 03:11:16.689472 14026 raft_consensus.cc:2781] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:16.689603 14026 raft_consensus.cc:482] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:16.689742 14026 raft_consensus.cc:3037] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:16.693259 14026 raft_consensus.cc:504] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.694057 14026 leader_election.cc:288] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 election: Requested vote from peers f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523), 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873)
I20260811 03:11:16.694664 13941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c02e56d74540558c8d71235d754cd5" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4863db35bebd4c578e6f6c9e043227e3"
I20260811 03:11:16.694918 13941 raft_consensus.cc:3037] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:16.694962 13630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c02e56d74540558c8d71235d754cd5" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f699cf80b4f94fb0baae8cbf8e308269"
I20260811 03:11:16.695199 13630 raft_consensus.cc:3037] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:16.697816 13941 raft_consensus.cc:2454] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 1.
I20260811 03:11:16.698243 13709 leader_election.cc:302] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475f04462fe74bcdbdd8eb1545642544, 4863db35bebd4c578e6f6c9e043227e3; no voters:
I20260811 03:11:16.698421 13630 raft_consensus.cc:2454] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 1.
I20260811 03:11:16.698607 14026 raft_consensus.cc:2781] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:16.698823 14026 raft_consensus.cc:686] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 1 LEADER]: Becoming Leader. State: Replica: 475f04462fe74bcdbdd8eb1545642544, State: Running, Role: LEADER
I20260811 03:11:16.699100 14026 consensus_queue.cc:237] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:16.701795 13460 catalog_manager.cc:5495] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 reported cstate change: term changed from 0 to 1, leader changed from <none> to 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2). New cstate: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } health_report { overall_health: UNKNOWN } } }
I20260811 03:11:16.737742 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.729451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.11.60.2:37977)
0811 03:11:16.729650 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.729656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.729676 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:11:16.729771 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.729776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.730017 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.730164 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:16.730895 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.732273 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.734693 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.735510 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.735629 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.736539 (+ 910us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:16.736779 (+ 240us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:16.736865 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.737287 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.737372 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 03:11:16.757066 14074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.749637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:37977 (local address 127.11.60.2:48367)
0811 03:11:16.749979 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:16.750002 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:16.750174 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:16.750790 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:16.750794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:16.750806 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:11:16.751131 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.751143 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.752748 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.752754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.754504 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:16.754520 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.755571 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.755578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:16.755740 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.755787 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:11:16.756409 (+ 622us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:11:16.756421 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 03:11:16.756517 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 03:11:16.756744 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":48}
I20260811 03:11:16.757182 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:16.749378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:48367 (local address 127.11.60.2:37977)
0811 03:11:16.750083 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:16.750088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:16.750109 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:11:16.750420 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:16.750426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:16.750553 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:16.750695 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:16.751323 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.752599 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.754657 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:16.755395 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:16.755536 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:16.756002 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:16.756283 (+ 281us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:16.756384 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:16.756569 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 03:11:16.756708 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":572}
I20260811 03:11:16.767079 13941 raft_consensus.cc:1261] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Refusing update from remote peer 475f04462fe74bcdbdd8eb1545642544: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:16.767421 13630 raft_consensus.cc:1261] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Refusing update from remote peer 475f04462fe74bcdbdd8eb1545642544: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:16.767935 14068 consensus_queue.cc:1035] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 03:11:16.768406 14026 consensus_queue.cc:1035] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 03:11:16.812901 14072 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 03:11:18.443799 13460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:18.444020 13460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:18.448249 13460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df: Sending DeleteTablet for 3 replicas of tablet 55c02e56d74540558c8d71235d754cd5
I20260811 03:11:18.449138 13761 tablet_service.cc:1514] Processing DeleteTablet for tablet 55c02e56d74540558c8d71235d754cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:18 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:34074
I20260811 03:11:18.449316 13605 tablet_service.cc:1514] Processing DeleteTablet for tablet 55c02e56d74540558c8d71235d754cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:18 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:60454
I20260811 03:11:18.449610 13921 tablet_service.cc:1514] Processing DeleteTablet for tablet 55c02e56d74540558c8d71235d754cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:18 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:48156
I20260811 03:11:18.449934 14112 tablet_replica.cc:330] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544: stopping tablet replica
I20260811 03:11:18.450212 14112 raft_consensus.cc:2229] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:11:18.450850 14112 raft_consensus.cc:2258] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:18.458530 14112 ts_tablet_manager.cc:1905] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:18.459153 14114 tablet_replica.cc:330] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3: stopping tablet replica
I20260811 03:11:18.459564 14114 raft_consensus.cc:2229] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:18.459980 14114 raft_consensus.cc:2258] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:18.461527 14114 ts_tablet_manager.cc:1905] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:18.463064 14113 tablet_replica.cc:330] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269: stopping tablet replica
I20260811 03:11:18.463300 14113 raft_consensus.cc:2229] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:18.463697 14113 raft_consensus.cc:2258] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:18.464819 14113 ts_tablet_manager.cc:1905] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:18.469205 14112 ts_tablet_manager.cc:1918] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 03:11:18.469411 14112 log.cc:1196] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/55c02e56d74540558c8d71235d754cd5
I20260811 03:11:18.469616 14114 ts_tablet_manager.cc:1918] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 03:11:18.469834 14114 log.cc:1196] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/55c02e56d74540558c8d71235d754cd5
I20260811 03:11:18.470217 14112 ts_tablet_manager.cc:1939] T 55c02e56d74540558c8d71235d754cd5 P 475f04462fe74bcdbdd8eb1545642544: Deleting consensus metadata
I20260811 03:11:18.470490 14114 ts_tablet_manager.cc:1939] T 55c02e56d74540558c8d71235d754cd5 P 4863db35bebd4c578e6f6c9e043227e3: Deleting consensus metadata
I20260811 03:11:18.471635 13442 catalog_manager.cc:4841] TS 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2:37977): tablet 55c02e56d74540558c8d71235d754cd5 (table test-table [id=0dfec52230de405cb493540b07fead01]) successfully deleted
I20260811 03:11:18.471787 13444 catalog_manager.cc:4841] TS 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873): tablet 55c02e56d74540558c8d71235d754cd5 (table test-table [id=0dfec52230de405cb493540b07fead01]) successfully deleted
I20260811 03:11:18.472455 14113 ts_tablet_manager.cc:1918] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 03:11:18.472604 14113 log.cc:1196] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/55c02e56d74540558c8d71235d754cd5
I20260811 03:11:18.473069 14113 ts_tablet_manager.cc:1939] T 55c02e56d74540558c8d71235d754cd5 P f699cf80b4f94fb0baae8cbf8e308269: Deleting consensus metadata
I20260811 03:11:18.474107 13444 catalog_manager.cc:4841] TS f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523): tablet 55c02e56d74540558c8d71235d754cd5 (table test-table [id=0dfec52230de405cb493540b07fead01]) successfully deleted
Aug 11 03:11:18 dist-test-slave-tm1g krb5kdc[13412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417878, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 03:11:18 dist-test-slave-tm1g krb5kdc[13412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417878, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:18.514609 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:18.498175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59068 (local address 127.11.60.62:46435)
0811 03:11:18.499061 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:18.499066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:18.499141 (+ 75us) server_negotiation.cc:407] Connection header received
0811 03:11:18.499315 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:18.499319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:18.499461 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:18.499560 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:18.501057 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.501894 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.503329 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.503766 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.503877 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:18.508471 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:18.508476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:18.508480 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:18.508516 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:18.511653 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:18.512402 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:18.512415 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:18.512417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:18.512548 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:18.512929 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:18.512933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:18.512935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:18.513228 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:18.513499 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:18.513808 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 03:11:18.514077 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":421,"threads_started":1}
I20260811 03:11:18.518198 13460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:11:18.518923 13460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:18.525586 13605 tablet_service.cc:1467] Processing CreateTablet for tablet e8168fe3d6a8401cbed69d2604941f0c (DEFAULT_TABLE table=test-table [id=41a61a2655e24e03826ded04eec2191d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:18.525803 13761 tablet_service.cc:1467] Processing CreateTablet for tablet e8168fe3d6a8401cbed69d2604941f0c (DEFAULT_TABLE table=test-table [id=41a61a2655e24e03826ded04eec2191d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:18.525952 13921 tablet_service.cc:1467] Processing CreateTablet for tablet e8168fe3d6a8401cbed69d2604941f0c (DEFAULT_TABLE table=test-table [id=41a61a2655e24e03826ded04eec2191d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:18.526109 13605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8168fe3d6a8401cbed69d2604941f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:18.526271 13761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8168fe3d6a8401cbed69d2604941f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:18.526396 13921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8168fe3d6a8401cbed69d2604941f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:18.531934 14123 tablet_bootstrap.cc:492] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544: Bootstrap starting.
I20260811 03:11:18.533845 14123 tablet_bootstrap.cc:654] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:18.534036 14124 tablet_bootstrap.cc:492] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269: Bootstrap starting.
I20260811 03:11:18.534806 14125 tablet_bootstrap.cc:492] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3: Bootstrap starting.
I20260811 03:11:18.536456 14125 tablet_bootstrap.cc:654] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:18.537864 14124 tablet_bootstrap.cc:654] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:18.539395 14125 tablet_bootstrap.cc:492] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3: No bootstrap required, opened a new log
I20260811 03:11:18.539561 14123 tablet_bootstrap.cc:492] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544: No bootstrap required, opened a new log
I20260811 03:11:18.539645 14125 ts_tablet_manager.cc:1397] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 03:11:18.539732 14123 ts_tablet_manager.cc:1397] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 03:11:18.540531 14125 raft_consensus.cc:348] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.540603 14123 raft_consensus.cc:348] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.540827 14125 raft_consensus.cc:374] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:18.540835 14123 raft_consensus.cc:374] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:18.540925 14124 tablet_bootstrap.cc:492] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269: No bootstrap required, opened a new log
I20260811 03:11:18.540932 14123 raft_consensus.cc:729] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475f04462fe74bcdbdd8eb1545642544, State: Initialized, Role: FOLLOWER
I20260811 03:11:18.540962 14125 raft_consensus.cc:729] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4863db35bebd4c578e6f6c9e043227e3, State: Initialized, Role: FOLLOWER
I20260811 03:11:18.541231 14124 ts_tablet_manager.cc:1397] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260811 03:11:18.541265 14125 consensus_queue.cc:260] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.541324 14123 consensus_queue.cc:260] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.542532 14123 ts_tablet_manager.cc:1428] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:18.543395 14124 raft_consensus.cc:348] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.543641 14124 raft_consensus.cc:374] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:18.543778 14124 raft_consensus.cc:729] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f699cf80b4f94fb0baae8cbf8e308269, State: Initialized, Role: FOLLOWER
I20260811 03:11:18.544083 14124 consensus_queue.cc:260] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.545271 14124 ts_tablet_manager.cc:1428] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 03:11:18.545987 14125 ts_tablet_manager.cc:1428] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 03:11:18.564751 14129 raft_consensus.cc:482] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:18.565088 14129 raft_consensus.cc:504] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.567070 14129 leader_election.cc:288] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523), 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2:37977)
I20260811 03:11:18.572629 14026 raft_consensus.cc:482] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:18.572839 14026 raft_consensus.cc:504] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.573774 14026 leader_election.cc:288] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523), 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873)
I20260811 03:11:18.574380 13941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8168fe3d6a8401cbed69d2604941f0c" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4863db35bebd4c578e6f6c9e043227e3" is_pre_election: true
I20260811 03:11:18.574668 13941 raft_consensus.cc:2454] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 0.
I20260811 03:11:18.575449 13709 leader_election.cc:302] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475f04462fe74bcdbdd8eb1545642544, 4863db35bebd4c578e6f6c9e043227e3; no voters:
I20260811 03:11:18.575971 14026 raft_consensus.cc:2781] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:18.576164 14026 raft_consensus.cc:482] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:18.576354 14026 raft_consensus.cc:3037] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:18.581110 14026 raft_consensus.cc:504] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.582813 13941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8168fe3d6a8401cbed69d2604941f0c" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4863db35bebd4c578e6f6c9e043227e3"
I20260811 03:11:18.583134 13941 raft_consensus.cc:3037] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:18.586565 13630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8168fe3d6a8401cbed69d2604941f0c" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f699cf80b4f94fb0baae8cbf8e308269" is_pre_election: true
I20260811 03:11:18.586736 13629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8168fe3d6a8401cbed69d2604941f0c" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f699cf80b4f94fb0baae8cbf8e308269"
I20260811 03:11:18.587067 13630 raft_consensus.cc:2454] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 0.
I20260811 03:11:18.588320 13941 raft_consensus.cc:2454] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 1.
I20260811 03:11:18.588744 13709 leader_election.cc:302] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475f04462fe74bcdbdd8eb1545642544, 4863db35bebd4c578e6f6c9e043227e3; no voters:
I20260811 03:11:18.589071 14098 raft_consensus.cc:2781] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:18.589285 14098 raft_consensus.cc:686] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 1 LEADER]: Becoming Leader. State: Replica: 475f04462fe74bcdbdd8eb1545642544, State: Running, Role: LEADER
I20260811 03:11:18.590235 14098 consensus_queue.cc:237] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.590806 14026 leader_election.cc:288] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 election: Requested vote from peers f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523), 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873)
I20260811 03:11:18.594233 13457 catalog_manager.cc:5495] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 reported cstate change: term changed from 0 to 1, leader changed from <none> to 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2). New cstate: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } health_report { overall_health: UNKNOWN } } }
I20260811 03:11:18.595559 14132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:18.567364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:45999 (local address 127.11.60.1:37523)
0811 03:11:18.585527 (+ 18163us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:18.585536 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:18.585563 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:11:18.585636 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:18.585643 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:18.585719 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:18.587263 (+ 1544us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:18.587531 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.588879 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.592507 (+ 3628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.593963 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.594117 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:18.594401 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:18.594844 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 03:11:18.595009 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17986,"thread_start_us":18443,"threads_started":1}
I20260811 03:11:18.595953 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:18.567321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:37523 (local address 127.11.60.3:45999)
0811 03:11:18.581033 (+ 13712us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:18.581058 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:18.581207 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:18.586517 (+ 5310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:18.586525 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:18.586559 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:18.586959 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:18.586974 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.589553 (+ 2579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.589558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:18.592273 (+ 2715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:18.592290 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.594283 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.594295 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:18.594610 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:18.594693 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 03:11:18.594819 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 03:11:18.594991 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13548,"thread_start_us":22328,"threads_started":1}
I20260811 03:11:18.596753 13630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8168fe3d6a8401cbed69d2604941f0c" candidate_uuid: "4863db35bebd4c578e6f6c9e043227e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f699cf80b4f94fb0baae8cbf8e308269" is_pre_election: true
I20260811 03:11:18.597055 13630 raft_consensus.cc:2454] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4863db35bebd4c578e6f6c9e043227e3 in term 0.
I20260811 03:11:18.597659 13868 leader_election.cc:302] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4863db35bebd4c578e6f6c9e043227e3, f699cf80b4f94fb0baae8cbf8e308269; no voters:
I20260811 03:11:18.598127 14129 raft_consensus.cc:2741] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 03:11:18.599538 14135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:18.567541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:52689 (local address 127.11.60.2:37977)
0811 03:11:18.584687 (+ 17146us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:18.584695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:18.590685 (+ 5990us) server_negotiation.cc:407] Connection header received
0811 03:11:18.590768 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:18.590774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:18.590878 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:18.591053 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:18.592080 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.593681 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.596790 (+ 3109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.598048 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.598190 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:18.598373 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:18.598982 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 03:11:18.599155 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16971,"thread_start_us":16809,"threads_started":1}
I20260811 03:11:18.600119 14131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:18.567326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:37977 (local address 127.11.60.3:52689)
0811 03:11:18.590502 (+ 23176us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:18.590523 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:18.590656 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:18.591215 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:18.591221 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:18.591259 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:18.591828 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:18.591841 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.593833 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.593840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:18.596606 (+ 2766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:18.596624 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.598586 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.598594 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:18.598781 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:18.598822 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 03:11:18.599476 (+ 654us) client_negotiation.cc:240] Negotiation successful
0811 03:11:18.599643 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23039,"mutex_wait_us":49,"thread_start_us":22737,"threads_started":1}
I20260811 03:11:18.600723 13789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8168fe3d6a8401cbed69d2604941f0c" candidate_uuid: "4863db35bebd4c578e6f6c9e043227e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "475f04462fe74bcdbdd8eb1545642544" is_pre_election: true
I20260811 03:11:18.617336 14135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:18.611944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40614 (local address 127.11.60.2:37977)
0811 03:11:18.612131 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:18.612137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:18.612269 (+ 132us) server_negotiation.cc:407] Connection header received
0811 03:11:18.612505 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:18.612510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:18.612589 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:18.612687 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:18.613072 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.613812 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.615241 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:18.615906 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:18.616049 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:18.616535 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:18.616640 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:18.616743 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:18.616901 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 03:11:18.616984 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 03:11:18.627106 13630 raft_consensus.cc:3037] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:18.627481 13941 raft_consensus.cc:1261] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Refusing update from remote peer 475f04462fe74bcdbdd8eb1545642544: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:18.628147 14026 consensus_queue.cc:1035] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:18.630483 13630 raft_consensus.cc:1261] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Refusing update from remote peer 475f04462fe74bcdbdd8eb1545642544: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:18.631070 14026 consensus_queue.cc:1035] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:18.674075 13458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:18.674384 13458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:18.677376 13458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df: Sending DeleteTablet for 3 replicas of tablet e8168fe3d6a8401cbed69d2604941f0c
I20260811 03:11:18.678319 13605 tablet_service.cc:1514] Processing DeleteTablet for tablet e8168fe3d6a8401cbed69d2604941f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:18 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:60454
I20260811 03:11:18.678417 13760 tablet_service.cc:1514] Processing DeleteTablet for tablet e8168fe3d6a8401cbed69d2604941f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:18 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:34074
I20260811 03:11:18.678426 13921 tablet_service.cc:1514] Processing DeleteTablet for tablet e8168fe3d6a8401cbed69d2604941f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:18 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:48156
I20260811 03:11:18.678699 14113 tablet_replica.cc:330] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269: stopping tablet replica
I20260811 03:11:18.678764 14114 tablet_replica.cc:330] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3: stopping tablet replica
I20260811 03:11:18.678987 14113 raft_consensus.cc:2229] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:18.679009 14114 raft_consensus.cc:2229] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:18.679231 14114 raft_consensus.cc:2258] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:18.679301 14113 raft_consensus.cc:2258] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:18.679651 14112 tablet_replica.cc:330] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544: stopping tablet replica
I20260811 03:11:18.679876 14112 raft_consensus.cc:2229] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:11:18.680174 14112 raft_consensus.cc:2258] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:18.680315 14114 ts_tablet_manager.cc:1905] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:18.681094 14112 ts_tablet_manager.cc:1905] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:18.682615 13458 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:11:18.683288 13458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:18.685226 14113 ts_tablet_manager.cc:1905] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:18.686378 14114 ts_tablet_manager.cc:1918] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:11:18.686566 14114 log.cc:1196] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/e8168fe3d6a8401cbed69d2604941f0c
I20260811 03:11:18.686997 14114 ts_tablet_manager.cc:1939] T e8168fe3d6a8401cbed69d2604941f0c P 4863db35bebd4c578e6f6c9e043227e3: Deleting consensus metadata
I20260811 03:11:18.688060 13444 catalog_manager.cc:4841] TS 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873): tablet e8168fe3d6a8401cbed69d2604941f0c (table test-table [id=41a61a2655e24e03826ded04eec2191d]) successfully deleted
I20260811 03:11:18.690600 13760 tablet_service.cc:1467] Processing CreateTablet for tablet 77afa33206964c50b506d02c621d68bf (DEFAULT_TABLE table=test-table [id=bf7a58fd2a014da8b8cd40f050033360]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:18.690817 13921 tablet_service.cc:1467] Processing CreateTablet for tablet 77afa33206964c50b506d02c621d68bf (DEFAULT_TABLE table=test-table [id=bf7a58fd2a014da8b8cd40f050033360]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:18.691075 13760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77afa33206964c50b506d02c621d68bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:18.691382 13921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77afa33206964c50b506d02c621d68bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:18.694231 14113 ts_tablet_manager.cc:1918] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:11:18.694428 14113 log.cc:1196] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/e8168fe3d6a8401cbed69d2604941f0c
I20260811 03:11:18.694928 14113 ts_tablet_manager.cc:1939] T e8168fe3d6a8401cbed69d2604941f0c P f699cf80b4f94fb0baae8cbf8e308269: Deleting consensus metadata
I20260811 03:11:18.695804 14123 tablet_bootstrap.cc:492] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544: Bootstrap starting.
I20260811 03:11:18.697885 14123 tablet_bootstrap.cc:654] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:18.698498 14125 tablet_bootstrap.cc:492] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3: Bootstrap starting.
I20260811 03:11:18.698887 13444 catalog_manager.cc:4841] TS f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523): tablet e8168fe3d6a8401cbed69d2604941f0c (table test-table [id=41a61a2655e24e03826ded04eec2191d]) successfully deleted
I20260811 03:11:18.699085 13605 tablet_service.cc:1467] Processing CreateTablet for tablet 77afa33206964c50b506d02c621d68bf (DEFAULT_TABLE table=test-table [id=bf7a58fd2a014da8b8cd40f050033360]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:18.699653 13605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77afa33206964c50b506d02c621d68bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:18.700879 14125 tablet_bootstrap.cc:654] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:18.704946 14125 tablet_bootstrap.cc:492] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3: No bootstrap required, opened a new log
I20260811 03:11:18.705216 14125 ts_tablet_manager.cc:1397] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 03:11:18.698571 14112 ts_tablet_manager.cc:1918] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:11:18.706035 14124 tablet_bootstrap.cc:492] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269: Bootstrap starting.
I20260811 03:11:18.706128 14112 log.cc:1196] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/e8168fe3d6a8401cbed69d2604941f0c
I20260811 03:11:18.706919 14112 ts_tablet_manager.cc:1939] T e8168fe3d6a8401cbed69d2604941f0c P 475f04462fe74bcdbdd8eb1545642544: Deleting consensus metadata
I20260811 03:11:18.707255 14125 raft_consensus.cc:348] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.707566 14125 raft_consensus.cc:374] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:18.707647 14125 raft_consensus.cc:729] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4863db35bebd4c578e6f6c9e043227e3, State: Initialized, Role: FOLLOWER
I20260811 03:11:18.707898 14125 consensus_queue.cc:260] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.708335 14124 tablet_bootstrap.cc:654] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:18.708680 14125 ts_tablet_manager.cc:1428] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:18.709712 14123 tablet_bootstrap.cc:492] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544: No bootstrap required, opened a new log
I20260811 03:11:18.709776 13442 catalog_manager.cc:4841] TS 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2:37977): tablet e8168fe3d6a8401cbed69d2604941f0c (table test-table [id=41a61a2655e24e03826ded04eec2191d]) successfully deleted
I20260811 03:11:18.709890 14123 ts_tablet_manager.cc:1397] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.009s
I20260811 03:11:18.710999 14123 raft_consensus.cc:348] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.711360 14123 raft_consensus.cc:374] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:18.711521 14123 raft_consensus.cc:729] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475f04462fe74bcdbdd8eb1545642544, State: Initialized, Role: FOLLOWER
I20260811 03:11:18.711905 14124 tablet_bootstrap.cc:492] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269: No bootstrap required, opened a new log
I20260811 03:11:18.711833 14123 consensus_queue.cc:260] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.712049 14124 ts_tablet_manager.cc:1397] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 03:11:18.712957 14123 ts_tablet_manager.cc:1428] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:18.712826 14124 raft_consensus.cc:348] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.713200 14124 raft_consensus.cc:374] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:18.713284 14124 raft_consensus.cc:729] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f699cf80b4f94fb0baae8cbf8e308269, State: Initialized, Role: FOLLOWER
I20260811 03:11:18.713460 14124 consensus_queue.cc:260] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.713986 14124 ts_tablet_manager.cc:1428] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 03:11:18.734283 14026 raft_consensus.cc:482] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:18.734539 14026 raft_consensus.cc:504] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.735140 14026 leader_election.cc:288] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523), 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873)
I20260811 03:11:18.735818 13630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77afa33206964c50b506d02c621d68bf" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f699cf80b4f94fb0baae8cbf8e308269" is_pre_election: true
I20260811 03:11:18.735952 13942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77afa33206964c50b506d02c621d68bf" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4863db35bebd4c578e6f6c9e043227e3" is_pre_election: true
I20260811 03:11:18.736105 13630 raft_consensus.cc:2454] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 0.
I20260811 03:11:18.736294 13942 raft_consensus.cc:2454] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 0.
I20260811 03:11:18.736672 13709 leader_election.cc:302] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475f04462fe74bcdbdd8eb1545642544, f699cf80b4f94fb0baae8cbf8e308269; no voters:
I20260811 03:11:18.737005 14026 raft_consensus.cc:2781] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:18.737221 14026 raft_consensus.cc:482] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:18.737422 14026 raft_consensus.cc:3037] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:18.740350 14026 raft_consensus.cc:504] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.741047 14026 leader_election.cc:288] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 election: Requested vote from peers f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523), 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873)
I20260811 03:11:18.742000 13630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77afa33206964c50b506d02c621d68bf" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f699cf80b4f94fb0baae8cbf8e308269"
I20260811 03:11:18.741974 13942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77afa33206964c50b506d02c621d68bf" candidate_uuid: "475f04462fe74bcdbdd8eb1545642544" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4863db35bebd4c578e6f6c9e043227e3"
I20260811 03:11:18.742233 13630 raft_consensus.cc:3037] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:18.742377 13942 raft_consensus.cc:3037] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:18.744985 13942 raft_consensus.cc:2454] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 1.
I20260811 03:11:18.745455 13709 leader_election.cc:302] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475f04462fe74bcdbdd8eb1545642544, 4863db35bebd4c578e6f6c9e043227e3; no voters:
I20260811 03:11:18.745896 14026 raft_consensus.cc:2781] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:18.746122 14026 raft_consensus.cc:686] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 1 LEADER]: Becoming Leader. State: Replica: 475f04462fe74bcdbdd8eb1545642544, State: Running, Role: LEADER
I20260811 03:11:18.746588 14026 consensus_queue.cc:237] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } }
I20260811 03:11:18.749511 13630 raft_consensus.cc:2454] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475f04462fe74bcdbdd8eb1545642544 in term 1.
I20260811 03:11:18.752156 13459 catalog_manager.cc:5495] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 reported cstate change: term changed from 0 to 1, leader changed from <none> to 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2). New cstate: current_term: 1 leader_uuid: "475f04462fe74bcdbdd8eb1545642544" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "475f04462fe74bcdbdd8eb1545642544" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 } health_report { overall_health: UNKNOWN } } }
I20260811 03:11:18.784184 13942 raft_consensus.cc:1261] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Refusing update from remote peer 475f04462fe74bcdbdd8eb1545642544: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:18.784533 13630 raft_consensus.cc:1261] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Refusing update from remote peer 475f04462fe74bcdbdd8eb1545642544: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:18.785061 14026 consensus_queue.cc:1035] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4863db35bebd4c578e6f6c9e043227e3" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 35873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:18.785499 14090 consensus_queue.cc:1035] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f699cf80b4f94fb0baae8cbf8e308269" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 03:11:18.811558 14091 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 03:11:20.394933 13458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:20.395212 13458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:20.399832 13458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38c0d65d6541405286c0d25d485911df: Sending DeleteTablet for 3 replicas of tablet 77afa33206964c50b506d02c621d68bf
I20260811 03:11:20.400713 13605 tablet_service.cc:1514] Processing DeleteTablet for tablet 77afa33206964c50b506d02c621d68bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:20 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:60454
I20260811 03:11:20.400732 13760 tablet_service.cc:1514] Processing DeleteTablet for tablet 77afa33206964c50b506d02c621d68bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:20 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:34074
I20260811 03:11:20.400790 13921 tablet_service.cc:1514] Processing DeleteTablet for tablet 77afa33206964c50b506d02c621d68bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:20 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:48156
I20260811 03:11:20.402536 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13536
I20260811 03:11:20.402894 14170 tablet_replica.cc:330] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544: stopping tablet replica
I20260811 03:11:20.403165 14170 raft_consensus.cc:2229] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:11:20.403609 14170 raft_consensus.cc:2258] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:20.404280 14171 tablet_replica.cc:330] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269: stopping tablet replica
I20260811 03:11:20.404521 14171 raft_consensus.cc:2229] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:20.404837 14171 raft_consensus.cc:2258] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:20.404889 14170 ts_tablet_manager.cc:1905] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:20.406804 14172 tablet_replica.cc:330] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3: stopping tablet replica
I20260811 03:11:20.407094 14172 raft_consensus.cc:2229] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:20.407245 14171 ts_tablet_manager.cc:1905] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:20.407423 14172 raft_consensus.cc:2258] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:20.408532 14172 ts_tablet_manager.cc:1905] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:20.412636 14170 ts_tablet_manager.cc:1918] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 03:11:20.412788 14170 log.cc:1196] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/77afa33206964c50b506d02c621d68bf
I20260811 03:11:20.413412 14170 ts_tablet_manager.cc:1939] T 77afa33206964c50b506d02c621d68bf P 475f04462fe74bcdbdd8eb1545642544: Deleting consensus metadata
I20260811 03:11:20.414609 13442 catalog_manager.cc:4841] TS 475f04462fe74bcdbdd8eb1545642544 (127.11.60.2:37977): tablet 77afa33206964c50b506d02c621d68bf (table test-table [id=bf7a58fd2a014da8b8cd40f050033360]) successfully deleted
I20260811 03:11:20.415154 14171 ts_tablet_manager.cc:1918] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 03:11:20.415310 14171 log.cc:1196] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/77afa33206964c50b506d02c621d68bf
I20260811 03:11:20.415513 14172 ts_tablet_manager.cc:1918] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 03:11:20.415688 14172 log.cc:1196] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/77afa33206964c50b506d02c621d68bf
I20260811 03:11:20.415967 14171 ts_tablet_manager.cc:1939] T 77afa33206964c50b506d02c621d68bf P f699cf80b4f94fb0baae8cbf8e308269: Deleting consensus metadata
I20260811 03:11:20.416098 14172 ts_tablet_manager.cc:1939] T 77afa33206964c50b506d02c621d68bf P 4863db35bebd4c578e6f6c9e043227e3: Deleting consensus metadata
I20260811 03:11:20.416986 13444 catalog_manager.cc:4841] TS 4863db35bebd4c578e6f6c9e043227e3 (127.11.60.3:35873): tablet 77afa33206964c50b506d02c621d68bf (table test-table [id=bf7a58fd2a014da8b8cd40f050033360]) successfully deleted
I20260811 03:11:20.417233 13444 catalog_manager.cc:4841] TS f699cf80b4f94fb0baae8cbf8e308269 (127.11.60.1:37523): tablet 77afa33206964c50b506d02c621d68bf (table test-table [id=bf7a58fd2a014da8b8cd40f050033360]) successfully deleted
Aug 11 03:11:20 dist-test-slave-tm1g krb5kdc[13412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417878, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:20.442240 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:20.403558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56634 (local address 127.11.60.1:37523)
0811 03:11:20.405538 (+ 1980us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:20.405547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:20.406166 (+ 619us) server_negotiation.cc:407] Connection header received
0811 03:11:20.406401 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:20.406410 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:20.406527 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:20.406642 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:20.408462 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:20.409948 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:20.411968 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:20.412704 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:20.412851 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:20.435781 (+ 22930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:20.435799 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:20.435816 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:20.435866 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:20.439317 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:20.440036 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:20.440044 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:20.440046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:20.440200 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:20.440616 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:20.440620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:20.440622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:20.441001 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:20.441248 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:20.441543 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 03:11:20.441809 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1810,"thread_start_us":1934,"threads_started":1}
W20260811 03:11:20.442988 13678 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56634
W20260811 03:11:20.443681 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13536: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:11:20.443903 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 13536
I20260811 03:11:20.470959 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13687
Aug 11 03:11:20 dist-test-slave-tm1g krb5kdc[13412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417878, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:11:20.487279 14175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:20.471700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.11.60.2:37977)
0811 03:11:20.472374 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:20.472381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:20.472405 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:11:20.472475 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:20.472479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:20.472546 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:20.472636 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:20.474137 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:20.475012 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:20.476268 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:20.476680 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:20.476780 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:20.481438 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:20.481452 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:20.481469 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:20.481508 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:20.484626 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:20.485266 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:20.485273 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:20.485275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:20.485414 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:20.485733 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:20.485736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:20.485738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:20.486109 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:20.486508 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:20.486688 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 03:11:20.486862 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":358,"threads_started":1}
W20260811 03:11:20.487937 13835 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40626
W20260811 03:11:20.488569 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13687: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:11:20.488699 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 13687
I20260811 03:11:20.519768 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13846
Aug 11 03:11:20 dist-test-slave-tm1g krb5kdc[13412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417878, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:11:20.536988 14177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:20.520591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44452 (local address 127.11.60.3:35873)
0811 03:11:20.521152 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:20.521156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:20.521173 (+ 17us) server_negotiation.cc:407] Connection header received
0811 03:11:20.521231 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:20.521235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:20.521298 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:20.521441 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:20.523132 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:20.523973 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:20.525383 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:20.525858 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:20.526026 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:20.530902 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:20.530919 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:20.530936 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:20.530981 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:20.534027 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:20.534818 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:20.534826 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:20.534829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:20.534988 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:20.535409 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:20.535413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:20.535415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:20.535765 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:20.536123 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:20.536396 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 03:11:20.536638 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":267,"threads_started":1}
W20260811 03:11:20.537711 13993 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44452
W20260811 03:11:20.538410 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13846: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:11:20.538568 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 13846
I20260811 03:11:20.559090 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 13425
I20260811 03:11:20.560556 13503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:20.683462 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 13425
2026-08-11T03:11:20Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8355 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:11:20 dist-test-slave-tm1g krb5kdc[14182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:11:20 dist-test-slave-tm1g krb5kdc[14182](info): set up 2 sockets
Aug 11 03:11:20 dist-test-slave-tm1g krb5kdc[14182](info): commencing operation
krb5kdc: starting...
W20260811 03:11:21.838618 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:11:21 dist-test-slave-tm1g krb5kdc[14182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:11:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:11:22.053987 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:44879
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:44879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:22.387204 14195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:22.387496 14195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:22.387588 14195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:22.394884 14195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:22.395006 14195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:22.395056 14195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:22.395112 14195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:22.395159 14195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:22.404964 14195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:44879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:44879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:22.406342 14195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:22.407632 14195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:22.421684 14200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:22.421656 14204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:22.421603 14201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:22.422204 14195 server_base.cc:1034] running on GCE node
I20260811 03:11:22.423255 14195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:22.424823 14195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:22.426076 14195 hybrid_clock.cc:648] HybridClock initialized: now 1786417882426044 us; error 57 us; skew 500 ppm
I20260811 03:11:22.426497 14195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:22 dist-test-slave-tm1g krb5kdc[14182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417882, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:22.429752 14195 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:11:22.431573 14195 webserver.cc:460] Webserver started at http://127.11.60.62:37255/ using document root <none> and password file <none>
I20260811 03:11:22.432253 14195 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:22.432354 14195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:22.432631 14195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:22.435665 14195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "8877de3d9a7c4163a7448fd6c16c98f6"
format_stamp: "Formatted at 2026-08-11 03:11:22 on dist-test-slave-tm1g"
server_key: "77fc89ba5617846f5542061c872ae85b"
server_key_iv: "6f255480e147ebafc1f9422e77e90114"
server_key_version: "encryptionkey@0"
I20260811 03:11:22.436380 14195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "8877de3d9a7c4163a7448fd6c16c98f6"
format_stamp: "Formatted at 2026-08-11 03:11:22 on dist-test-slave-tm1g"
server_key: "77fc89ba5617846f5542061c872ae85b"
server_key_iv: "6f255480e147ebafc1f9422e77e90114"
server_key_version: "encryptionkey@0"
I20260811 03:11:22.441632 14195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 03:11:22.445149 14211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:22.446542 14195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:22.446673 14195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "8877de3d9a7c4163a7448fd6c16c98f6"
format_stamp: "Formatted at 2026-08-11 03:11:22 on dist-test-slave-tm1g"
server_key: "77fc89ba5617846f5542061c872ae85b"
server_key_iv: "6f255480e147ebafc1f9422e77e90114"
server_key_version: "encryptionkey@0"
I20260811 03:11:22.446921 14195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:22.461685 14195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:22.466384 14195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:22.466660 14195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:22.487135 14195 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:44879
I20260811 03:11:22.487175 14269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:44879 every 8 connection(s)
I20260811 03:11:22.488699 14195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:22.492630 14270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:22.496227 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 14195
I20260811 03:11:22.496492 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:22.497295 11504 external_mini_cluster.cc:1428] Setting key 5dd6a3907c3dae457f682c36ad00c271
I20260811 03:11:22.505784 14270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6: Bootstrap starting.
I20260811 03:11:22.509008 14270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:22.510016 14270 log.cc:824] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6: Log is configured to *not* fsync() on all Append() calls
Aug 11 03:11:22 dist-test-slave-tm1g krb5kdc[14182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:22.513392 14270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6: No bootstrap required, opened a new log
I20260811 03:11:22.522027 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:22.499653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58280 (local address 127.11.60.62:44879)
0811 03:11:22.500467 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:22.500480 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:22.500523 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:11:22.502237 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:22.502260 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:22.502775 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:22.503211 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:22.504794 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:22.506220 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:22.507572 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:22.508273 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:22.508478 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:22.512752 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:22.512765 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:22.512783 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:22.512836 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:22.517037 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:22.517825 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:22.517835 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:22.517845 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:22.518070 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:22.518630 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:22.518634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:22.518638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:22.519408 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:22.519739 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:22.520080 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 03:11:22.520570 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":381,"threads_started":1}
I20260811 03:11:22.524426 14270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8877de3d9a7c4163a7448fd6c16c98f6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44879 } }
I20260811 03:11:22.524871 14270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:22.524996 14270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8877de3d9a7c4163a7448fd6c16c98f6, State: Initialized, Role: FOLLOWER
I20260811 03:11:22.525619 14270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8877de3d9a7c4163a7448fd6c16c98f6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44879 } }
I20260811 03:11:22.525840 14270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:22.525939 14270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:22.526062 14270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:22.528213 14270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8877de3d9a7c4163a7448fd6c16c98f6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44879 } }
I20260811 03:11:22.528702 14270 leader_election.cc:302] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8877de3d9a7c4163a7448fd6c16c98f6; no voters:
I20260811 03:11:22.529213 14270 leader_election.cc:288] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:22.529364 14276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:22.530596 14276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [term 1 LEADER]: Becoming Leader. State: Replica: 8877de3d9a7c4163a7448fd6c16c98f6, State: Running, Role: LEADER
I20260811 03:11:22.531101 14276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8877de3d9a7c4163a7448fd6c16c98f6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44879 } }
I20260811 03:11:22.531468 14270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:22.533737 14278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8877de3d9a7c4163a7448fd6c16c98f6. Latest consensus state: current_term: 1 leader_uuid: "8877de3d9a7c4163a7448fd6c16c98f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8877de3d9a7c4163a7448fd6c16c98f6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44879 } } }
I20260811 03:11:22.534068 14278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:22.534583 14277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8877de3d9a7c4163a7448fd6c16c98f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8877de3d9a7c4163a7448fd6c16c98f6" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44879 } } }
I20260811 03:11:22.534860 14277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:22.536834 14285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:22.542162 14285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:22.564951 14285 catalog_manager.cc:1347] Generated new cluster ID: 089a9df47a224ff098efb8beccf36cfa
I20260811 03:11:22.565093 14285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:22.583650 14285 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:22.585091 14285 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:22.599521 14285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8877de3d9a7c4163a7448fd6c16c98f6: Generated new TSK 0
I20260811 03:11:22.600394 14285 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:11:22.658826 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44879
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:22.999032 14301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:22.999398 14301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:22.999568 14301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:23.006954 14301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:23.007125 14301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:23.007314 14301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:23.017515 14301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:23.018990 14301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:23.020422 14301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:23.034910 14307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:23.035012 14310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:23.035020 14308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:23.036437 14301 server_base.cc:1034] running on GCE node
I20260811 03:11:23.037189 14301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:23.038280 14301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:23.039575 14301 hybrid_clock.cc:648] HybridClock initialized: now 1786417883039515 us; error 31 us; skew 500 ppm
I20260811 03:11:23.039937 14301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:23 dist-test-slave-tm1g krb5kdc[14182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417883, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:23.043251 14301 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:11:23.044729 14301 webserver.cc:460] Webserver started at http://127.11.60.1:45931/ using document root <none> and password file <none>
I20260811 03:11:23.045455 14301 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:23.045572 14301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:23.045838 14301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:23.048913 14301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "06cd4cd895b640419df5a37279faa387"
format_stamp: "Formatted at 2026-08-11 03:11:23 on dist-test-slave-tm1g"
server_key: "c1df870b057326caa9ab0649ca277e1f"
server_key_iv: "b3a9c8a429cb81750a12e86d4e3073b6"
server_key_version: "encryptionkey@0"
I20260811 03:11:23.049696 14301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "06cd4cd895b640419df5a37279faa387"
format_stamp: "Formatted at 2026-08-11 03:11:23 on dist-test-slave-tm1g"
server_key: "c1df870b057326caa9ab0649ca277e1f"
server_key_iv: "b3a9c8a429cb81750a12e86d4e3073b6"
server_key_version: "encryptionkey@0"
I20260811 03:11:23.055131 14301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 03:11:23.058877 14318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:23.060331 14301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 03:11:23.060520 14301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "06cd4cd895b640419df5a37279faa387"
format_stamp: "Formatted at 2026-08-11 03:11:23 on dist-test-slave-tm1g"
server_key: "c1df870b057326caa9ab0649ca277e1f"
server_key_iv: "b3a9c8a429cb81750a12e86d4e3073b6"
server_key_version: "encryptionkey@0"
I20260811 03:11:23.060716 14301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:23.088188 14301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:23.092880 14301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:23.093204 14301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:23.094283 14301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:23.095722 14301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:23.095830 14301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:23.095988 14301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:23.096067 14301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:23.130782 14301 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:34729
I20260811 03:11:23.130884 14449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:34729 every 8 connection(s)
I20260811 03:11:23.132359 14301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:23.142092 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 14301
I20260811 03:11:23.142390 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:23.143038 11504 external_mini_cluster.cc:1428] Setting key ebf5ad212f590ce083812c63e00d5435
Aug 11 03:11:23 dist-test-slave-tm1g krb5kdc[14182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417883, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:23.153755 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:23.135225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:42695 (local address 127.11.60.62:44879)
0811 03:11:23.135665 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:23.135669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:23.136763 (+ 1094us) server_negotiation.cc:407] Connection header received
0811 03:11:23.138920 (+ 2157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:23.138927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:23.139032 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:23.139157 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:23.140685 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:23.141412 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:23.143239 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:23.143652 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:23.143794 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:23.147777 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:23.147782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:23.147785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:23.147819 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:23.150612 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:23.151384 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:23.151389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:23.151391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:23.151518 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:23.151972 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:23.151981 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:23.151983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:23.152255 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:23.152415 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:23.153145 (+ 730us) server_negotiation.cc:299] Negotiation successful
0811 03:11:23.153377 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":213,"threads_started":1}
I20260811 03:11:23.154837 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:23.135800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44879 (local address 127.11.60.1:42695)
0811 03:11:23.136588 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:23.136640 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:23.138632 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:23.139407 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:23.139420 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:23.139932 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:23.140503 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:23.140521 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:23.141592 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:23.141597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:23.143053 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:23.143070 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:23.143884 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:23.143892 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:23.144140 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:23.144859 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:23.144882 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:23.147546 (+ 2664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:23.150763 (+ 3217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:23.150772 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:23.150788 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:23.151243 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:23.151631 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:23.151636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:23.151640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:23.151859 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:23.152389 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:23.152396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:23.152754 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 03:11:23.153024 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 03:11:23.153580 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":276,"threads_started":1}
I20260811 03:11:23.156463 14450 heartbeater.cc:347] Connected to a master server at 127.11.60.62:44879
I20260811 03:11:23.156800 14450 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:23.157527 14450 heartbeater.cc:511] Master 127.11.60.62:44879 requested a full tablet report, sending...
I20260811 03:11:23.159936 14231 ts_manager.cc:194] Registered new tserver with Master: 06cd4cd895b640419df5a37279faa387 (127.11.60.1:34729)
I20260811 03:11:23.162245 14231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:42695
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:11:23.194039 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44879
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:23.532186 14456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:23.532505 14456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:23.532642 14456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:23.540030 14456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:23.540136 14456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:23.540252 14456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:23.549871 14456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:23.551285 14456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:23.552579 14456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:23.566097 14464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:23.566126 14467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:23.566249 14465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:23.567129 14456 server_base.cc:1034] running on GCE node
I20260811 03:11:23.567765 14456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:23.568723 14456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:23.569967 14456 hybrid_clock.cc:648] HybridClock initialized: now 1786417883569917 us; error 61 us; skew 500 ppm
I20260811 03:11:23.570297 14456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:23 dist-test-slave-tm1g krb5kdc[14182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417883, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:23.573449 14456 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:11:23.574960 14456 webserver.cc:460] Webserver started at http://127.11.60.2:40253/ using document root <none> and password file <none>
I20260811 03:11:23.575672 14456 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:23.575817 14456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:23.576139 14456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:23.579129 14456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "c2289d96030d44dbba2f88d137e5fcf4"
format_stamp: "Formatted at 2026-08-11 03:11:23 on dist-test-slave-tm1g"
server_key: "f30af6171739fb1c48cf7cf8590fc6d0"
server_key_iv: "21fca29b8a47594ca9fd048268484702"
server_key_version: "encryptionkey@0"
I20260811 03:11:23.579931 14456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "c2289d96030d44dbba2f88d137e5fcf4"
format_stamp: "Formatted at 2026-08-11 03:11:23 on dist-test-slave-tm1g"
server_key: "f30af6171739fb1c48cf7cf8590fc6d0"
server_key_iv: "21fca29b8a47594ca9fd048268484702"
server_key_version: "encryptionkey@0"
I20260811 03:11:23.585139 14456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:11:23.588608 14474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:23.589908 14456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 03:11:23.590101 14456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "c2289d96030d44dbba2f88d137e5fcf4"
format_stamp: "Formatted at 2026-08-11 03:11:23 on dist-test-slave-tm1g"
server_key: "f30af6171739fb1c48cf7cf8590fc6d0"
server_key_iv: "21fca29b8a47594ca9fd048268484702"
server_key_version: "encryptionkey@0"
I20260811 03:11:23.590438 14456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:23.605635 14456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:23.609589 14456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:23.609881 14456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:23.610927 14456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:23.612253 14456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:23.612357 14456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:23.612501 14456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:23.612581 14456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:23.646739 14456 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:43517
I20260811 03:11:23.646793 14605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:43517 every 8 connection(s)
I20260811 03:11:23.648098 14456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:23.656152 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 14456
I20260811 03:11:23.656405 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:23.656912 11504 external_mini_cluster.cc:1428] Setting key d920dc3d3d13d13662e556d27325ecfa
Aug 11 03:11:23 dist-test-slave-tm1g krb5kdc[14182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417883, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:23.671602 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:23.650734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:47345 (local address 127.11.60.62:44879)
0811 03:11:23.650906 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:23.650911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:23.652266 (+ 1355us) server_negotiation.cc:407] Connection header received
0811 03:11:23.654223 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:23.654228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:23.654347 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:23.654458 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:23.656170 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:23.657110 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:23.659348 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:23.659794 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:23.659907 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:23.664821 (+ 4914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:23.664827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:23.664830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:23.664865 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:23.667759 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:23.668808 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:23.668814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:23.668819 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:23.668991 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:23.669489 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:23.669498 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:23.669504 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:23.669886 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:23.670051 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:23.670846 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 03:11:23.671118 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 03:11:23.672392 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:23.651174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44879 (local address 127.11.60.2:47345)
0811 03:11:23.652061 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:23.652124 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:23.653936 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:23.654661 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:23.654669 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:23.655214 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:23.655973 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:23.656004 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:23.657291 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:23.657298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:23.659198 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:23.659215 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:23.660044 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:23.660055 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:23.660414 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:23.661207 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:23.661246 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:23.664550 (+ 3304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:23.667916 (+ 3366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:23.667928 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:23.667949 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:23.668666 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:23.669120 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:23.669124 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:23.669126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:23.669320 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:23.670014 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:23.670019 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:23.670474 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 03:11:23.670730 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 03:11:23.671139 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":351,"threads_started":1}
I20260811 03:11:23.674017 14606 heartbeater.cc:347] Connected to a master server at 127.11.60.62:44879
I20260811 03:11:23.674460 14606 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:23.675259 14606 heartbeater.cc:511] Master 127.11.60.62:44879 requested a full tablet report, sending...
I20260811 03:11:23.676884 14231 ts_manager.cc:194] Registered new tserver with Master: c2289d96030d44dbba2f88d137e5fcf4 (127.11.60.2:43517)
I20260811 03:11:23.677786 14231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:47345
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:11:23.707949 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44879
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:24.044759 14612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:24.045092 14612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:24.045264 14612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:24.052739 14612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:24.052887 14612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:24.053072 14612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:24.062808 14612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:24.064214 14612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:24.065706 14612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:24.079895 14621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:24.079905 14619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:24.079946 14618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:24.080653 14612 server_base.cc:1034] running on GCE node
I20260811 03:11:24.081303 14612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:24.082334 14612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:24.083551 14612 hybrid_clock.cc:648] HybridClock initialized: now 1786417884083502 us; error 44 us; skew 500 ppm
I20260811 03:11:24.083873 14612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:24 dist-test-slave-tm1g krb5kdc[14182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417884, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:24.087143 14612 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:11:24.088801 14612 webserver.cc:460] Webserver started at http://127.11.60.3:40679/ using document root <none> and password file <none>
I20260811 03:11:24.089576 14612 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:24.089725 14612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:24.090054 14612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:24.093060 14612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "d59676ecc2e84392a51b781319ddfa23"
format_stamp: "Formatted at 2026-08-11 03:11:24 on dist-test-slave-tm1g"
server_key: "e2969a6b2487fd86a834891d1c02db44"
server_key_iv: "f9e88384574af8cb9fa0b6ecfd7286b1"
server_key_version: "encryptionkey@0"
I20260811 03:11:24.093955 14612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "d59676ecc2e84392a51b781319ddfa23"
format_stamp: "Formatted at 2026-08-11 03:11:24 on dist-test-slave-tm1g"
server_key: "e2969a6b2487fd86a834891d1c02db44"
server_key_iv: "f9e88384574af8cb9fa0b6ecfd7286b1"
server_key_version: "encryptionkey@0"
I20260811 03:11:24.099357 14612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 03:11:24.102799 14631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:24.104194 14612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 03:11:24.104386 14612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "d59676ecc2e84392a51b781319ddfa23"
format_stamp: "Formatted at 2026-08-11 03:11:24 on dist-test-slave-tm1g"
server_key: "e2969a6b2487fd86a834891d1c02db44"
server_key_iv: "f9e88384574af8cb9fa0b6ecfd7286b1"
server_key_version: "encryptionkey@0"
I20260811 03:11:24.104614 14612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:24.128984 14612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:24.133761 14612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:24.134083 14612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:24.135092 14612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:24.136379 14612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:24.136458 14612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:24.136643 14612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:24.136703 14612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:24.165832 14450 heartbeater.cc:503] Master 127.11.60.62:44879 was elected leader, sending a full tablet report...
I20260811 03:11:24.172086 14612 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:42537
I20260811 03:11:24.172108 14770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:42537 every 8 connection(s)
I20260811 03:11:24.173458 14612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:24.179307 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 14612
I20260811 03:11:24.179534 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:24.179986 11504 external_mini_cluster.cc:1428] Setting key c8bcb0410eadd7ac821ea3373628f16e
Aug 11 03:11:24 dist-test-slave-tm1g krb5kdc[14182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417884, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:24.193832 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:24.176157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:52409 (local address 127.11.60.62:44879)
0811 03:11:24.176702 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:24.176707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:24.177530 (+ 823us) server_negotiation.cc:407] Connection header received
0811 03:11:24.179554 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:24.179560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:24.179666 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:24.179797 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:24.181471 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:24.182377 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:24.183950 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:24.184346 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:24.184490 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:24.188154 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:24.188159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:24.188162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:24.188197 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:24.190947 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:24.191664 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:24.191668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:24.191670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:24.191793 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:24.192166 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:24.192169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:24.192171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:24.192437 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:24.192546 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:24.193250 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 03:11:24.193440 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":328,"threads_started":1}
I20260811 03:11:24.195078 14774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:24.176598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44879 (local address 127.11.60.3:52409)
0811 03:11:24.177339 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:24.177382 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:24.179247 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:24.180045 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:24.180054 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:24.180608 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:24.181301 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:24.181322 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:24.182532 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:24.182536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:24.183762 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:24.183775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:24.184578 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:24.184587 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:24.184822 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:24.185528 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:24.185552 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:24.187951 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:24.191087 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:24.191095 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:24.191107 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:24.191556 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:24.191892 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:24.191895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:24.191897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:24.192074 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:24.192541 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:24.192546 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:24.192844 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 03:11:24.193091 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 03:11:24.193620 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":309,"threads_started":1}
I20260811 03:11:24.196439 14771 heartbeater.cc:347] Connected to a master server at 127.11.60.62:44879
I20260811 03:11:24.196750 14771 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:24.197458 14771 heartbeater.cc:511] Master 127.11.60.62:44879 requested a full tablet report, sending...
I20260811 03:11:24.198892 14231 ts_manager.cc:194] Registered new tserver with Master: d59676ecc2e84392a51b781319ddfa23 (127.11.60.3:42537)
I20260811 03:11:24.199730 14231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:52409
I20260811 03:11:24.208204 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 03:11:24.246676 14773 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:11:24.244032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58290 (local address 127.11.60.62:44879)
0811 03:11:24.244278 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:24.244287 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:24.244725 (+ 438us) server_negotiation.cc:407] Connection header received
0811 03:11:24.244957 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:24.244961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:24.245033 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:24.245144 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:24.246477 (+ 1333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58290: BlockingRecv error: recv got EOF from 127.0.0.1:58290 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 03:11:24.250363 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 14301
W20260811 03:11:24.253520 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 14301: Not authorized: Client connection negotiation failed: client connection to 127.11.60.1:34729: server requires authentication, but client does not have Kerberos credentials available
I20260811 03:11:24.253712 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 14301
I20260811 03:11:24.272060 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 14456
W20260811 03:11:24.274958 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 14456: Not authorized: Client connection negotiation failed: client connection to 127.11.60.2:43517: server requires authentication, but client does not have Kerberos credentials available
I20260811 03:11:24.275142 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 14456
I20260811 03:11:24.294341 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 14612
W20260811 03:11:24.297627 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 14612: Not authorized: Client connection negotiation failed: client connection to 127.11.60.3:42537: server requires authentication, but client does not have Kerberos credentials available
I20260811 03:11:24.297750 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 14612
I20260811 03:11:24.316238 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 14195
I20260811 03:11:24.317549 14265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:24.424503 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 14195
2026-08-11T03:11:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3737 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:11:24 dist-test-slave-tm1g krb5kdc[14789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:11:24 dist-test-slave-tm1g krb5kdc[14789](info): set up 2 sockets
Aug 11 03:11:24 dist-test-slave-tm1g krb5kdc[14789](info): commencing operation
krb5kdc: starting...
W20260811 03:11:25.577805 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:11:25 dist-test-slave-tm1g krb5kdc[14789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:11:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:11:25.805434 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42143
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:36627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:42143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:26.133301 14803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:26.133576 14803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:26.133665 14803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:26.140772 14803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:26.140875 14803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:26.140921 14803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:26.140970 14803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:26.141018 14803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:26.150745 14803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:36627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:42143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:26.151970 14803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:26.153257 14803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:26.165928 14809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:26.165966 14813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:26.165872 14810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:26.166916 14803 server_base.cc:1034] running on GCE node
I20260811 03:11:26.167711 14803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:26.169111 14803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:26.170403 14803 hybrid_clock.cc:648] HybridClock initialized: now 1786417886170361 us; error 49 us; skew 500 ppm
I20260811 03:11:26.170825 14803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:26 dist-test-slave-tm1g krb5kdc[14789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417886, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:26.179585 14803 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:11:26.181190 14803 webserver.cc:460] Webserver started at http://127.11.60.62:42103/ using document root <none> and password file <none>
I20260811 03:11:26.181933 14803 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:26.182088 14803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:26.182502 14803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:26.185736 14803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "6d9b181949244622af9ce67474f935bb"
format_stamp: "Formatted at 2026-08-11 03:11:26 on dist-test-slave-tm1g"
server_key: "8d796ecfb1de47cb7b0f927b1712484d"
server_key_iv: "06fb253b82bca5eabda836cfe46778c2"
server_key_version: "encryptionkey@0"
I20260811 03:11:26.186681 14803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "6d9b181949244622af9ce67474f935bb"
format_stamp: "Formatted at 2026-08-11 03:11:26 on dist-test-slave-tm1g"
server_key: "8d796ecfb1de47cb7b0f927b1712484d"
server_key_iv: "06fb253b82bca5eabda836cfe46778c2"
server_key_version: "encryptionkey@0"
I20260811 03:11:26.191890 14803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:11:26.195319 14821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:26.196614 14803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 03:11:26.196798 14803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "6d9b181949244622af9ce67474f935bb"
format_stamp: "Formatted at 2026-08-11 03:11:26 on dist-test-slave-tm1g"
server_key: "8d796ecfb1de47cb7b0f927b1712484d"
server_key_iv: "06fb253b82bca5eabda836cfe46778c2"
server_key_version: "encryptionkey@0"
I20260811 03:11:26.197001 14803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:26.220563 14803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:26.225345 14803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:26.225673 14803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:26.245460 14803 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:42143
I20260811 03:11:26.245497 14877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:42143 every 8 connection(s)
I20260811 03:11:26.246932 14803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:26.250597 14878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:26.255491 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 14803
I20260811 03:11:26.255743 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:26.256563 11504 external_mini_cluster.cc:1428] Setting key a75344e59bf46de15125b8513d386267
I20260811 03:11:26.262113 14878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb: Bootstrap starting.
I20260811 03:11:26.265905 14878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:26.267124 14878 log.cc:824] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:26.270519 14878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb: No bootstrap required, opened a new log
Aug 11 03:11:26 dist-test-slave-tm1g krb5kdc[14789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:26.278131 14878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b181949244622af9ce67474f935bb" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42143 } }
I20260811 03:11:26.278625 14878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:26.278756 14878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d9b181949244622af9ce67474f935bb, State: Initialized, Role: FOLLOWER
I20260811 03:11:26.279342 14878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b181949244622af9ce67474f935bb" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42143 } }
I20260811 03:11:26.279599 14878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:26.279752 14878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:26.279963 14878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:26.280018 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:26.258505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39186 (local address 127.11.60.62:42143)
0811 03:11:26.259339 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:26.259350 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:26.259395 (+ 45us) server_negotiation.cc:407] Connection header received
0811 03:11:26.261085 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:26.261107 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:26.261460 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:26.261755 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:26.263658 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:26.264963 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:26.266810 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:26.267527 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:26.267757 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:26.272430 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:26.272440 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:26.272455 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:26.272487 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:26.275682 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:26.276397 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:26.276403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:26.276412 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:26.276588 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:26.277004 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:26.277008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:26.277010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:26.277552 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:26.277857 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:26.278342 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 03:11:26.278792 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":392,"threads_started":1}
I20260811 03:11:26.282366 14878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b181949244622af9ce67474f935bb" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42143 } }
I20260811 03:11:26.282930 14878 leader_election.cc:302] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d9b181949244622af9ce67474f935bb; no voters:
I20260811 03:11:26.283546 14878 leader_election.cc:288] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:26.283694 14883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:26.283949 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:11:26.284346 14883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [term 1 LEADER]: Becoming Leader. State: Replica: 6d9b181949244622af9ce67474f935bb, State: Running, Role: LEADER
I20260811 03:11:26.284965 14883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b181949244622af9ce67474f935bb" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42143 } }
I20260811 03:11:26.287453 14885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d9b181949244622af9ce67474f935bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b181949244622af9ce67474f935bb" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42143 } } }
I20260811 03:11:26.287827 14885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:26.290002 14885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d9b181949244622af9ce67474f935bb. Latest consensus state: current_term: 1 leader_uuid: "6d9b181949244622af9ce67474f935bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b181949244622af9ce67474f935bb" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42143 } } }
I20260811 03:11:26.290341 14885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:26.295223 14878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:26.295441 14888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:26.301110 14888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:26.310357 14888 catalog_manager.cc:1347] Generated new cluster ID: 0cf3f999fbb24e459888e23c7b67bfdf
I20260811 03:11:26.310468 14888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:26.326529 14888 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:26.328480 14888 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:26.343494 14888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d9b181949244622af9ce67474f935bb: Generated new TSK 0
I20260811 03:11:26.344607 14888 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:11:26.352748 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:26.345107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.11.60.62:42143)
0811 03:11:26.345265 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:26.345272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:26.345882 (+ 610us) server_negotiation.cc:407] Connection header received
0811 03:11:26.346141 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:26.346147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:26.346222 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:26.346356 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:26.347430 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:26.348185 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:26.349487 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:26.349900 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:26.350047 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:26.351241 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:26.351245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:26.351249 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:26.351989 (+ 740us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:26.352046 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:26.352074 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:26.352200 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:26.352318 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 03:11:26.352417 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 03:11:26.360183 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 14803
I20260811 03:11:26.361312 14873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:26.452529 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 14803
2026-08-11T03:11:26Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2023 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:11:26 dist-test-slave-tm1g krb5kdc[14918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:11:26 dist-test-slave-tm1g krb5kdc[14918](info): set up 2 sockets
Aug 11 03:11:26 dist-test-slave-tm1g krb5kdc[14918](info): commencing operation
krb5kdc: starting...
W20260811 03:11:27.632426 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:11:27 dist-test-slave-tm1g krb5kdc[14918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:11:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:11:27.879840 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:37913
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:37913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:28.211773 14931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:28.212145 14931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:28.212317 14931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:28.219539 14931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:28.219704 14931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:28.219818 14931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:28.219935 14931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:28.220022 14931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:28.230129 14931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:37913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:37913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:28.231597 14931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:28.233130 14931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:28.245893 14936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:28.245843 14938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:28.245815 14940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:28.246663 14931 server_base.cc:1034] running on GCE node
I20260811 03:11:28.247381 14931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:28.248699 14931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:28.249927 14931 hybrid_clock.cc:648] HybridClock initialized: now 1786417888249876 us; error 38 us; skew 500 ppm
I20260811 03:11:28.250385 14931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:28 dist-test-slave-tm1g krb5kdc[14918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417888, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:28.253809 14931 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:11:28.255483 14931 webserver.cc:460] Webserver started at http://127.11.60.62:45739/ using document root <none> and password file <none>
I20260811 03:11:28.256192 14931 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:28.256343 14931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:28.256693 14931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:28.259897 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "3dc03e56bf514840a4737ab7677d1b5c"
format_stamp: "Formatted at 2026-08-11 03:11:28 on dist-test-slave-tm1g"
server_key: "fedb2423a16ecdcbbad54e845271de29"
server_key_iv: "ee7c0aa8afeed434a67340c19f26de96"
server_key_version: "encryptionkey@0"
I20260811 03:11:28.260675 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "3dc03e56bf514840a4737ab7677d1b5c"
format_stamp: "Formatted at 2026-08-11 03:11:28 on dist-test-slave-tm1g"
server_key: "fedb2423a16ecdcbbad54e845271de29"
server_key_iv: "ee7c0aa8afeed434a67340c19f26de96"
server_key_version: "encryptionkey@0"
I20260811 03:11:28.266064 14931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 03:11:28.269459 14947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:28.270835 14931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:28.271029 14931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "3dc03e56bf514840a4737ab7677d1b5c"
format_stamp: "Formatted at 2026-08-11 03:11:28 on dist-test-slave-tm1g"
server_key: "fedb2423a16ecdcbbad54e845271de29"
server_key_iv: "ee7c0aa8afeed434a67340c19f26de96"
server_key_version: "encryptionkey@0"
I20260811 03:11:28.271229 14931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:28.287968 14931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:28.292222 14931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:28.292542 14931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:28.312021 14931 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:37913
I20260811 03:11:28.312059 15004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:37913 every 8 connection(s)
I20260811 03:11:28.313433 14931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:28.317466 15005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:28.320919 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 14931
I20260811 03:11:28.321161 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:28.321799 11504 external_mini_cluster.cc:1428] Setting key d4f10e098b44e7e190ff64ae785bf403
I20260811 03:11:28.329967 15005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c: Bootstrap starting.
I20260811 03:11:28.333160 15005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:28.334175 15005 log.cc:824] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c: Log is configured to *not* fsync() on all Append() calls
Aug 11 03:11:28 dist-test-slave-tm1g krb5kdc[14918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:28.337510 15005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c: No bootstrap required, opened a new log
I20260811 03:11:28.346218 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:28.323992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48812 (local address 127.11.60.62:37913)
0811 03:11:28.324780 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:28.324792 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:28.324835 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:11:28.326580 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:28.326603 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:28.327005 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:28.327415 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:28.328861 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:28.330224 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:28.331502 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:28.332178 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:28.332384 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:28.336764 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:28.336776 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:28.336800 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:28.336848 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:28.341092 (+ 4244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:28.341896 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:28.341910 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:28.341919 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:28.342153 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:28.342721 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:28.342725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:28.342728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:28.343511 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:28.343838 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:28.344250 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 03:11:28.344749 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":360,"threads_started":1}
I20260811 03:11:28.348340 15005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc03e56bf514840a4737ab7677d1b5c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37913 } }
I20260811 03:11:28.348793 15005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:28.348906 15005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dc03e56bf514840a4737ab7677d1b5c, State: Initialized, Role: FOLLOWER
I20260811 03:11:28.349514 15005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc03e56bf514840a4737ab7677d1b5c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37913 } }
I20260811 03:11:28.349774 15005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:28.349931 15005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:28.350106 15005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:28.352314 15005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc03e56bf514840a4737ab7677d1b5c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37913 } }
I20260811 03:11:28.352815 15005 leader_election.cc:302] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3dc03e56bf514840a4737ab7677d1b5c; no voters:
I20260811 03:11:28.353461 15005 leader_election.cc:288] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:28.353626 15011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:28.355067 15011 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [term 1 LEADER]: Becoming Leader. State: Replica: 3dc03e56bf514840a4737ab7677d1b5c, State: Running, Role: LEADER
I20260811 03:11:28.355720 15005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:28.355602 15011 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc03e56bf514840a4737ab7677d1b5c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37913 } }
I20260811 03:11:28.358177 15013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3dc03e56bf514840a4737ab7677d1b5c. Latest consensus state: current_term: 1 leader_uuid: "3dc03e56bf514840a4737ab7677d1b5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc03e56bf514840a4737ab7677d1b5c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37913 } } }
I20260811 03:11:28.358582 15013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:28.358610 15012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3dc03e56bf514840a4737ab7677d1b5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc03e56bf514840a4737ab7677d1b5c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37913 } } }
I20260811 03:11:28.358914 15012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:28.366480 15018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:28.371254 15018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:28.379751 15018 catalog_manager.cc:1347] Generated new cluster ID: bac5e79e0ce14de99ae046f17d6b12eb
I20260811 03:11:28.379855 15018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:28.406972 15018 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:28.408453 15018 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:28.416535 15018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3dc03e56bf514840a4737ab7677d1b5c: Generated new TSK 0
I20260811 03:11:28.417356 15018 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:11:28.473765 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37913
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:28.831017 15032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:28.831339 15032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:28.831507 15032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:28.838861 15032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:28.839025 15032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:28.839198 15032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:28.849697 15032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:28.851163 15032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:28.852530 15032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:28.866014 15037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:28.866331 15038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:28.866199 15041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:28.867728 15032 server_base.cc:1034] running on GCE node
I20260811 03:11:28.868319 15032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:28.869266 15032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:28.870486 15032 hybrid_clock.cc:648] HybridClock initialized: now 1786417888870426 us; error 53 us; skew 500 ppm
I20260811 03:11:28.870790 15032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:28 dist-test-slave-tm1g krb5kdc[14918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417888, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:28.873955 15032 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:11:28.875526 15032 webserver.cc:460] Webserver started at http://127.11.60.1:34397/ using document root <none> and password file <none>
I20260811 03:11:28.876261 15032 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:28.876410 15032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:28.876719 15032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:28.879750 15032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "cdd4ca3b7fb14e0da06e76566abe14bb"
format_stamp: "Formatted at 2026-08-11 03:11:28 on dist-test-slave-tm1g"
server_key: "512a49785e1e2372c793e7210986422e"
server_key_iv: "6122f0a51dbf78b187233fec5581d465"
server_key_version: "encryptionkey@0"
I20260811 03:11:28.880541 15032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "cdd4ca3b7fb14e0da06e76566abe14bb"
format_stamp: "Formatted at 2026-08-11 03:11:28 on dist-test-slave-tm1g"
server_key: "512a49785e1e2372c793e7210986422e"
server_key_iv: "6122f0a51dbf78b187233fec5581d465"
server_key_version: "encryptionkey@0"
I20260811 03:11:28.885612 15032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:11:28.888974 15048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:28.890249 15032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:11:28.890434 15032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "cdd4ca3b7fb14e0da06e76566abe14bb"
format_stamp: "Formatted at 2026-08-11 03:11:28 on dist-test-slave-tm1g"
server_key: "512a49785e1e2372c793e7210986422e"
server_key_iv: "6122f0a51dbf78b187233fec5581d465"
server_key_version: "encryptionkey@0"
I20260811 03:11:28.890622 15032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:28.918689 15032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:28.923007 15032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:28.923328 15032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:28.924283 15032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:28.925665 15032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:28.925769 15032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:28.925925 15032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:28.925999 15032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:28.959894 15032 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:38027
I20260811 03:11:28.959944 15168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:38027 every 8 connection(s)
I20260811 03:11:28.961246 15032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:28.968631 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 15032
I20260811 03:11:28.968863 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:28.969717 11504 external_mini_cluster.cc:1428] Setting key 7b00635274340958edb9cd0b23ac6804
Aug 11 03:11:28 dist-test-slave-tm1g krb5kdc[14918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417888, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:28.984017 15171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:28.964080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:47139 (local address 127.11.60.62:37913)
0811 03:11:28.964560 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:28.964566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:28.965671 (+ 1105us) server_negotiation.cc:407] Connection header received
0811 03:11:28.967511 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:28.967516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:28.967589 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:28.967703 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:28.969334 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:28.970493 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:28.972267 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:28.972974 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:28.973132 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:28.977065 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:28.977074 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:28.977082 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:28.977131 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:28.980382 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:28.981331 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:28.981339 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:28.981345 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:28.981500 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:28.981914 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:28.981922 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:28.981935 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:28.982353 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:28.982503 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:28.983227 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 03:11:28.983497 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":233,"threads_started":1}
I20260811 03:11:28.984684 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:28.964718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37913 (local address 127.11.60.1:47139)
0811 03:11:28.965493 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:28.965540 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:28.967235 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:28.967901 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:28.967910 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:28.968457 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:28.969143 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:28.969163 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:28.970668 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:28.970674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:28.972106 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:28.972118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:28.973197 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:28.973203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:28.973399 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:28.974192 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:28.974256 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:28.976798 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:28.980538 (+ 3740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:28.980549 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:28.980591 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:28.981207 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:28.981628 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:28.981631 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:28.981633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:28.981824 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:28.982489 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:28.982495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:28.982830 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 03:11:28.983074 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 03:11:28.983497 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":317,"threads_started":1}
I20260811 03:11:28.986423 15169 heartbeater.cc:347] Connected to a master server at 127.11.60.62:37913
I20260811 03:11:28.986768 15169 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:28.987723 15169 heartbeater.cc:511] Master 127.11.60.62:37913 requested a full tablet report, sending...
I20260811 03:11:28.990176 14968 ts_manager.cc:194] Registered new tserver with Master: cdd4ca3b7fb14e0da06e76566abe14bb (127.11.60.1:38027)
I20260811 03:11:28.991680 14968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:47139
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:11:29.022620 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37913
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:29.359984 15175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:29.360316 15175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:29.360492 15175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:29.368130 15175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:29.368283 15175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:29.368469 15175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:29.379138 15175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:29.380535 15175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:29.381929 15175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:29.395463 15180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:29.395481 15183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:29.395457 15181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:29.396688 15175 server_base.cc:1034] running on GCE node
I20260811 03:11:29.397276 15175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:29.398190 15175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:29.399425 15175 hybrid_clock.cc:648] HybridClock initialized: now 1786417889399391 us; error 62 us; skew 500 ppm
I20260811 03:11:29.399740 15175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:29 dist-test-slave-tm1g krb5kdc[14918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417889, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:29.402910 15175 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:11:29.404420 15175 webserver.cc:460] Webserver started at http://127.11.60.2:43867/ using document root <none> and password file <none>
I20260811 03:11:29.405110 15175 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:29.405261 15175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:29.405601 15175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:29.408516 15175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "6ff7d186aa9446bdb471d4bdf34ebbb5"
format_stamp: "Formatted at 2026-08-11 03:11:29 on dist-test-slave-tm1g"
server_key: "740bc4a1ed003d6d5b44605ac02a4f74"
server_key_iv: "cea8ad7bfe7aa4a931906a971454efcc"
server_key_version: "encryptionkey@0"
I20260811 03:11:29.409317 15175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "6ff7d186aa9446bdb471d4bdf34ebbb5"
format_stamp: "Formatted at 2026-08-11 03:11:29 on dist-test-slave-tm1g"
server_key: "740bc4a1ed003d6d5b44605ac02a4f74"
server_key_iv: "cea8ad7bfe7aa4a931906a971454efcc"
server_key_version: "encryptionkey@0"
I20260811 03:11:29.414528 15175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 03:11:29.418015 15191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:29.419345 15175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:29.419538 15175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "6ff7d186aa9446bdb471d4bdf34ebbb5"
format_stamp: "Formatted at 2026-08-11 03:11:29 on dist-test-slave-tm1g"
server_key: "740bc4a1ed003d6d5b44605ac02a4f74"
server_key_iv: "cea8ad7bfe7aa4a931906a971454efcc"
server_key_version: "encryptionkey@0"
I20260811 03:11:29.419739 15175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:29.439615 15175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:29.443594 15175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:29.443889 15175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:29.444859 15175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:29.446242 15175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:29.446372 15175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 03:11:29.446529 15175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:29.446604 15175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:29.481623 15175 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:36029
I20260811 03:11:29.481674 15311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:36029 every 8 connection(s)
I20260811 03:11:29.483006 15175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:29.484622 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 15175
I20260811 03:11:29.484814 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:29.485301 11504 external_mini_cluster.cc:1428] Setting key 5e21ee8bc72a1747716e4a70ea00655e
Aug 11 03:11:29 dist-test-slave-tm1g krb5kdc[14918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417889, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:29.503547 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:29.485751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:35979 (local address 127.11.60.62:37913)
0811 03:11:29.486591 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:29.486597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:29.487453 (+ 856us) server_negotiation.cc:407] Connection header received
0811 03:11:29.489055 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:29.489060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:29.489150 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:29.489268 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:29.490807 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:29.491557 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:29.493095 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:29.493539 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:29.493674 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:29.497737 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:29.497742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:29.497746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:29.497777 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:29.500339 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:29.501061 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:29.501072 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:29.501075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:29.501196 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:29.501638 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:29.501642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:29.501646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:29.501916 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:29.502074 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:29.502893 (+ 819us) server_negotiation.cc:299] Negotiation successful
0811 03:11:29.503130 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":570,"threads_started":1}
I20260811 03:11:29.504513 15315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:29.486134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37913 (local address 127.11.60.2:35979)
0811 03:11:29.487078 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:29.487137 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:29.488787 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:29.489503 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:29.489511 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:29.490013 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:29.490639 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:29.490657 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:29.491711 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:29.491717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:29.492944 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:29.492956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:29.493768 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:29.493776 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:29.494043 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:29.494890 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:29.494930 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:29.497510 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:29.500479 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:29.500488 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:29.500504 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:29.500924 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:29.501316 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:29.501321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:29.501324 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:29.501517 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:29.502052 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:29.502059 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:29.502496 (+ 437us) client_negotiation.cc:769] Sending connection context
0811 03:11:29.502792 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 03:11:29.503355 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":416,"threads_started":1}
I20260811 03:11:29.506099 15312 heartbeater.cc:347] Connected to a master server at 127.11.60.62:37913
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
I20260811 03:11:29.506443 15312 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:29.507258 15312 heartbeater.cc:511] Master 127.11.60.62:37913 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
I20260811 03:11:29.508942 14968 ts_manager.cc:194] Registered new tserver with Master: 6ff7d186aa9446bdb471d4bdf34ebbb5 (127.11.60.2:36029)
I20260811 03:11:29.510131 14968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:35979
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:11:29.535670 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37913
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:29.864636 15319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:29.864903 15319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:29.865056 15319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:29.872226 15319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:29.872328 15319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:29.872526 15319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:29.882212 15319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:29.883526 15319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:29.884802 15319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:29.897826 15327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:29.897962 15325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:29.897931 15324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:29.898896 15319 server_base.cc:1034] running on GCE node
I20260811 03:11:29.899474 15319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:29.900365 15319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:29.901587 15319 hybrid_clock.cc:648] HybridClock initialized: now 1786417889901505 us; error 69 us; skew 500 ppm
I20260811 03:11:29.901875 15319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:29 dist-test-slave-tm1g krb5kdc[14918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417889, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:29.905004 15319 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:11:29.906509 15319 webserver.cc:460] Webserver started at http://127.11.60.3:35697/ using document root <none> and password file <none>
I20260811 03:11:29.907169 15319 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:29.907313 15319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:29.907620 15319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:29.910483 15319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "13bd9cb4fc744f29ad6567f7b432db39"
format_stamp: "Formatted at 2026-08-11 03:11:29 on dist-test-slave-tm1g"
server_key: "b2a75436b8e238e3287cb2499ea5eef3"
server_key_iv: "00365899e13d9edc63851a06f55b0bc6"
server_key_version: "encryptionkey@0"
I20260811 03:11:29.911248 15319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "13bd9cb4fc744f29ad6567f7b432db39"
format_stamp: "Formatted at 2026-08-11 03:11:29 on dist-test-slave-tm1g"
server_key: "b2a75436b8e238e3287cb2499ea5eef3"
server_key_iv: "00365899e13d9edc63851a06f55b0bc6"
server_key_version: "encryptionkey@0"
I20260811 03:11:29.916103 15319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 03:11:29.919363 15334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:29.920552 15319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.004s
I20260811 03:11:29.920729 15319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "13bd9cb4fc744f29ad6567f7b432db39"
format_stamp: "Formatted at 2026-08-11 03:11:29 on dist-test-slave-tm1g"
server_key: "b2a75436b8e238e3287cb2499ea5eef3"
server_key_iv: "00365899e13d9edc63851a06f55b0bc6"
server_key_version: "encryptionkey@0"
I20260811 03:11:29.920925 15319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:29.936272 15319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:29.940130 15319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:29.940421 15319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:29.941339 15319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:29.942668 15319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:29.942742 15319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:29.942906 15319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:29.942961 15319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:29.975442 15319 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:43703
I20260811 03:11:29.975500 15459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:43703 every 8 connection(s)
I20260811 03:11:29.976742 15319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:29.985060 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 15319
I20260811 03:11:29.985251 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:29.985920 11504 external_mini_cluster.cc:1428] Setting key 988d7e1c92c812c902569863b48fc4d9
Aug 11 03:11:29 dist-test-slave-tm1g krb5kdc[14918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417889, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:29.995191 15169 heartbeater.cc:503] Master 127.11.60.62:37913 was elected leader, sending a full tablet report...
I20260811 03:11:29.997069 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:29.979191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:50963 (local address 127.11.60.62:37913)
0811 03:11:29.979406 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:29.979413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:29.980655 (+ 1242us) server_negotiation.cc:407] Connection header received
0811 03:11:29.982427 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:29.982431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:29.982541 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:29.982630 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:29.984448 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:29.985221 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:29.987378 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:29.987796 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:29.987934 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:29.991491 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:29.991496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:29.991499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:29.991530 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:29.993875 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:29.994701 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:29.994705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:29.994707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:29.994824 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:29.995266 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:29.995270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:29.995272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:29.995650 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:29.995833 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:29.996567 (+ 734us) server_negotiation.cc:299] Negotiation successful
0811 03:11:29.996845 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:11:29.997998 15462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:29.979746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37913 (local address 127.11.60.3:50963)
0811 03:11:29.980499 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:29.980534 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:29.982135 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:29.982830 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:29.982837 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:29.983330 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:29.984251 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:29.984275 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:29.985431 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:29.985439 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:29.987236 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:29.987249 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:29.988031 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:29.988038 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:29.988273 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:29.988972 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:29.988994 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:29.991290 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:29.993998 (+ 2708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:29.994004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:29.994014 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:29.994578 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:29.994919 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:29.994922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:29.994925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:29.995128 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:29.995823 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:29.995831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:29.996178 (+ 347us) client_negotiation.cc:769] Sending connection context
0811 03:11:29.996389 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 03:11:29.996900 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":283,"threads_started":1}
I20260811 03:11:29.999460 15460 heartbeater.cc:347] Connected to a master server at 127.11.60.62:37913
I20260811 03:11:29.999766 15460 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:30.000631 15460 heartbeater.cc:511] Master 127.11.60.62:37913 requested a full tablet report, sending...
I20260811 03:11:30.001999 14968 ts_manager.cc:194] Registered new tserver with Master: 13bd9cb4fc744f29ad6567f7b432db39 (127.11.60.3:43703)
I20260811 03:11:30.002868 14968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:50963
I20260811 03:11:30.013123 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 03:11:30 dist-test-slave-tm1g krb5kdc[14918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417890, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 03:11:30 dist-test-slave-tm1g krb5kdc[14918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417890, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:30.065765 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:30.051420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.11.60.62:37913)
0811 03:11:30.051614 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:30.051619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:30.052088 (+ 469us) server_negotiation.cc:407] Connection header received
0811 03:11:30.052235 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:30.052240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:30.052343 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:30.052440 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:30.053791 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:30.054636 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:30.055961 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:30.056424 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:30.056521 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:30.060488 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:30.060493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:30.060496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:30.060529 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:30.063245 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:30.063914 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:30.063917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:30.063919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:30.064050 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:30.064479 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:30.064482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:30.064484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:30.064789 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:30.064983 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:30.065289 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 03:11:30.065493 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 03:11:30.066377 14968 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52186
I20260811 03:11:30.068248 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 15032
Aug 11 03:11:30 dist-test-slave-tm1g krb5kdc[14918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417890, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:30.084345 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:30.069284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37134 (local address 127.11.60.1:38027)
0811 03:11:30.069938 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:30.069946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:30.069968 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:11:30.070021 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:30.070027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:30.070246 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:30.070533 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:30.071681 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:30.072684 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:30.073893 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:30.074571 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:30.074708 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:30.078626 (+ 3918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:30.078637 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:30.078651 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:30.078687 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:30.081500 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:30.082159 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:30.082172 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:30.082175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:30.082352 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:30.082739 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:30.082745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:30.082749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:30.083107 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:30.083397 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:30.083716 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 03:11:30.083989 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":333,"threads_started":1}
W20260811 03:11:30.085680 15164 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37134
W20260811 03:11:30.086818 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 15032: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:11:30.087059 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 15032
I20260811 03:11:30.106204 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 15175
Aug 11 03:11:30 dist-test-slave-tm1g krb5kdc[14918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417890, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:11:30.123081 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:30.107263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.11.60.2:36029)
0811 03:11:30.107842 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:30.107850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:30.107876 (+ 26us) server_negotiation.cc:407] Connection header received
0811 03:11:30.107936 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:30.107942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:30.108156 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:30.108294 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:30.109643 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:30.110824 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:30.112017 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:30.112619 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:30.112756 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:30.116899 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:30.116910 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:30.116924 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:30.116968 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:30.120143 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:30.120934 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:30.120941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:30.120943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:30.121085 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:30.121466 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:30.121469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:30.121471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:30.121815 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:30.122044 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:30.122455 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 03:11:30.122707 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":295,"threads_started":1}
W20260811 03:11:30.124485 15307 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55514
W20260811 03:11:30.125713 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 15175: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:11:30.125866 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 15175
I20260811 03:11:30.144361 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 15319
Aug 11 03:11:30 dist-test-slave-tm1g krb5kdc[14918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417890, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:11:30.161566 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:30.145597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59442 (local address 127.11.60.3:43703)
0811 03:11:30.146273 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:30.146281 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:30.146359 (+ 78us) server_negotiation.cc:407] Connection header received
0811 03:11:30.146436 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:30.146443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:30.146665 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:30.146800 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:30.148108 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:30.149193 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:30.150504 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:30.151061 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:30.151159 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:30.155454 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:30.155464 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:30.155477 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:30.155509 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:30.158678 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:30.159345 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:30.159352 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:30.159354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:30.159505 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:30.159876 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:30.159882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:30.159885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:30.160227 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:30.160564 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:30.160851 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 03:11:30.161133 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":355,"threads_started":1}
W20260811 03:11:30.163012 15455 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59442
W20260811 03:11:30.164180 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 15319: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:11:30.164353 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 15319
I20260811 03:11:30.184376 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 14931
I20260811 03:11:30.185678 15000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:30.292320 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 14931
2026-08-11T03:11:30Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3845 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:11:30 dist-test-slave-tm1g krb5kdc[15478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:11:30 dist-test-slave-tm1g krb5kdc[15478](info): set up 2 sockets
Aug 11 03:11:30 dist-test-slave-tm1g krb5kdc[15478](info): commencing operation
krb5kdc: starting...
W20260811 03:11:31.444751 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.006s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:11:31 dist-test-slave-tm1g krb5kdc[15478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:11:31.658090 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:45965
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:32773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:45965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:32.005446 15491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:32.005759 15491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:32.005880 15491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:32.013186 15491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:32.013298 15491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:32.013343 15491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:32.013399 15491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:32.013444 15491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:32.023440 15491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:32773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:45965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:45965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:32.024744 15491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:32.026036 15491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:32.038187 15496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:32.038126 15499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:32.038157 15497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:32.039248 15491 server_base.cc:1034] running on GCE node
I20260811 03:11:32.040009 15491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:32.041420 15491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:32.042626 15491 hybrid_clock.cc:648] HybridClock initialized: now 1786417892042580 us; error 34 us; skew 500 ppm
I20260811 03:11:32.042992 15491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:32 dist-test-slave-tm1g krb5kdc[15478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417892, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:32.046080 15491 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:11:32.047652 15491 webserver.cc:460] Webserver started at http://127.11.60.62:43243/ using document root <none> and password file <none>
I20260811 03:11:32.048389 15491 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:32.048550 15491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:32.048936 15491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:32.052011 15491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "1a31c20f462e43dfa7b446d814899e73"
format_stamp: "Formatted at 2026-08-11 03:11:32 on dist-test-slave-tm1g"
server_key: "ea424be9344e80632c50e2e9ed37906e"
server_key_iv: "0bc8070d69e743620bf6e112373a558a"
server_key_version: "encryptionkey@0"
I20260811 03:11:32.052837 15491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "1a31c20f462e43dfa7b446d814899e73"
format_stamp: "Formatted at 2026-08-11 03:11:32 on dist-test-slave-tm1g"
server_key: "ea424be9344e80632c50e2e9ed37906e"
server_key_iv: "0bc8070d69e743620bf6e112373a558a"
server_key_version: "encryptionkey@0"
I20260811 03:11:32.058079 15491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 03:11:32.061460 15507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:32.063006 15491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:32.063205 15491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "1a31c20f462e43dfa7b446d814899e73"
format_stamp: "Formatted at 2026-08-11 03:11:32 on dist-test-slave-tm1g"
server_key: "ea424be9344e80632c50e2e9ed37906e"
server_key_iv: "0bc8070d69e743620bf6e112373a558a"
server_key_version: "encryptionkey@0"
I20260811 03:11:32.063422 15491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:32.077850 15491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:32.081988 15491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:32.082278 15491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:32.101856 15491 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:45965
I20260811 03:11:32.101894 15565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:45965 every 8 connection(s)
I20260811 03:11:32.103286 15491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:32.107195 15566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:32.108184 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 15491
I20260811 03:11:32.108439 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:32.109279 11504 external_mini_cluster.cc:1428] Setting key c06861c31e64aa49067ac8c3c71dba44
I20260811 03:11:32.121795 15566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73: Bootstrap starting.
Aug 11 03:11:32 dist-test-slave-tm1g krb5kdc[15478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:32.125624 15566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:32.126868 15566 log.cc:824] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:32.129776 15566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73: No bootstrap required, opened a new log
I20260811 03:11:32.133311 15569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:32.111514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59132 (local address 127.11.60.62:45965)
0811 03:11:32.112332 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:32.112344 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:32.112388 (+ 44us) server_negotiation.cc:407] Connection header received
0811 03:11:32.114035 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:32.114060 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:32.114523 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:32.114860 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:32.116715 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:32.118039 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:32.119936 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:32.120670 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:32.120820 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:32.125797 (+ 4977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:32.125809 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:32.125824 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:32.125874 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:32.129262 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:32.129970 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:32.129977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:32.129983 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:32.130148 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:32.130589 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:32.130593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:32.130595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:32.131121 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:32.131466 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:32.131880 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 03:11:32.132296 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":388,"threads_started":1}
I20260811 03:11:32.138197 15566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a31c20f462e43dfa7b446d814899e73" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45965 } }
I20260811 03:11:32.138629 15566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:32.138741 15566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a31c20f462e43dfa7b446d814899e73, State: Initialized, Role: FOLLOWER
I20260811 03:11:32.139392 15566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a31c20f462e43dfa7b446d814899e73" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45965 } }
I20260811 03:11:32.139688 15566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:32.139839 15566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:32.140030 15566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:32.142192 15566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a31c20f462e43dfa7b446d814899e73" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45965 } }
I20260811 03:11:32.142735 15566 leader_election.cc:302] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a31c20f462e43dfa7b446d814899e73; no voters:
I20260811 03:11:32.143366 15566 leader_election.cc:288] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:32.143507 15571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:32.144217 15571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [term 1 LEADER]: Becoming Leader. State: Replica: 1a31c20f462e43dfa7b446d814899e73, State: Running, Role: LEADER
I20260811 03:11:32.144788 15571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a31c20f462e43dfa7b446d814899e73" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45965 } }
I20260811 03:11:32.145370 15566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:32.148006 15573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a31c20f462e43dfa7b446d814899e73. Latest consensus state: current_term: 1 leader_uuid: "1a31c20f462e43dfa7b446d814899e73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a31c20f462e43dfa7b446d814899e73" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45965 } } }
I20260811 03:11:32.148299 15573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:32.149036 15572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a31c20f462e43dfa7b446d814899e73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a31c20f462e43dfa7b446d814899e73" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45965 } } }
I20260811 03:11:32.149415 15572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:32.151422 15578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:32.157851 15578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:32.166705 15578 catalog_manager.cc:1347] Generated new cluster ID: 0004ae3f447643feb4a0033c62cc2829
I20260811 03:11:32.166817 15578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:32.188742 15578 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:32.190217 15578 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:32.200031 15578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a31c20f462e43dfa7b446d814899e73: Generated new TSK 0
I20260811 03:11:32.200886 15578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:11:32.262259 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:45965
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:32773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:32.600257 15594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:32.600543 15594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:32.600703 15594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:32.608131 15594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:32.608250 15594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:32.608440 15594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:32.618432 15594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:32773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:45965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:32.619797 15594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:32.621160 15594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:32.633812 15599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:32.634732 15600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:32.635298 15602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:32.636053 15594 server_base.cc:1034] running on GCE node
I20260811 03:11:32.636665 15594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:32.637657 15594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:32.638887 15594 hybrid_clock.cc:648] HybridClock initialized: now 1786417892638837 us; error 66 us; skew 500 ppm
I20260811 03:11:32.639215 15594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:32 dist-test-slave-tm1g krb5kdc[15478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417892, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:32.642494 15594 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:11:32.643836 15594 webserver.cc:460] Webserver started at http://127.11.60.1:35367/ using document root <none> and password file <none>
I20260811 03:11:32.644490 15594 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:32.644587 15594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:32.644847 15594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:32.647892 15594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "a1d761d0fa5e4334abd3bb9baf8bc717"
format_stamp: "Formatted at 2026-08-11 03:11:32 on dist-test-slave-tm1g"
server_key: "a65bf32ae6682216a2f18c578c32343c"
server_key_iv: "13ccf453f34873e8ae03f5f13577f1b5"
server_key_version: "encryptionkey@0"
I20260811 03:11:32.648663 15594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "a1d761d0fa5e4334abd3bb9baf8bc717"
format_stamp: "Formatted at 2026-08-11 03:11:32 on dist-test-slave-tm1g"
server_key: "a65bf32ae6682216a2f18c578c32343c"
server_key_iv: "13ccf453f34873e8ae03f5f13577f1b5"
server_key_version: "encryptionkey@0"
I20260811 03:11:32.654297 15594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 03:11:32.657850 15609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:32.659157 15594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:32.659461 15594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "a1d761d0fa5e4334abd3bb9baf8bc717"
format_stamp: "Formatted at 2026-08-11 03:11:32 on dist-test-slave-tm1g"
server_key: "a65bf32ae6682216a2f18c578c32343c"
server_key_iv: "13ccf453f34873e8ae03f5f13577f1b5"
server_key_version: "encryptionkey@0"
I20260811 03:11:32.659665 15594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:32.675940 15594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:32.679772 15594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:32.680058 15594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:32.680990 15594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:32.682351 15594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:32.682449 15594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:32.682610 15594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:32.682684 15594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:32.716068 15594 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:34957
I20260811 03:11:32.716164 15732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:34957 every 8 connection(s)
I20260811 03:11:32.717433 15594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:32.725916 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 15594
I20260811 03:11:32.726161 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:32.726938 11504 external_mini_cluster.cc:1428] Setting key 8c71d900cc42083c88dba67da6181e16
Aug 11 03:11:32 dist-test-slave-tm1g krb5kdc[15478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417892, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:32.738837 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:32.720263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:55791 (local address 127.11.60.62:45965)
0811 03:11:32.720720 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:32.720726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:32.721743 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 03:11:32.723641 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:32.723646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:32.723713 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:32.723814 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:32.725254 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:32.726405 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:32.728196 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:32.728812 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:32.728948 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:32.733153 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:32.733158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:32.733162 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:32.733192 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:32.735790 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:32.736501 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:32.736508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:32.736512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:32.736642 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:32.737087 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:32.737093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:32.737096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:32.737355 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:32.737510 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:32.738198 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 03:11:32.738452 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":212,"threads_started":1}
I20260811 03:11:32.739840 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:32.720853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:45965 (local address 127.11.60.1:55791)
0811 03:11:32.721580 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:32.721623 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:32.723364 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:32.724024 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:32.724031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:32.724546 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:32.725105 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:32.725123 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:32.726592 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:32.726596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:32.728058 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:32.728071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:32.729083 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:32.729092 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:32.729422 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:32.730123 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:32.730157 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:32.732924 (+ 2767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:32.735935 (+ 3011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:32.735944 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:32.735959 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:32.736380 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:32.736755 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:32.736760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:32.736764 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:32.736958 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:32.737486 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:32.737493 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:32.737819 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 03:11:32.738097 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 03:11:32.738635 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":284,"threads_started":1}
I20260811 03:11:32.741302 15735 heartbeater.cc:347] Connected to a master server at 127.11.60.62:45965
I20260811 03:11:32.741600 15735 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:32.742354 15735 heartbeater.cc:511] Master 127.11.60.62:45965 requested a full tablet report, sending...
I20260811 03:11:32.744381 15525 ts_manager.cc:194] Registered new tserver with Master: a1d761d0fa5e4334abd3bb9baf8bc717 (127.11.60.1:34957)
I20260811 03:11:32.745973 15525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:55791
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:11:32.777603 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:45965
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:32773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:33.124130 15741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:33.124400 15741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:33.124502 15741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:33.131825 15741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:33.131933 15741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:33.132050 15741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:33.141729 15741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:32773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:45965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:33.142992 15741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:33.144253 15741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:33.157546 15746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:33.157487 15748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:33.157794 15750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:33.158624 15741 server_base.cc:1034] running on GCE node
I20260811 03:11:33.159261 15741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:33.160207 15741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:33.161475 15741 hybrid_clock.cc:648] HybridClock initialized: now 1786417893161440 us; error 48 us; skew 500 ppm
I20260811 03:11:33.161791 15741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:33 dist-test-slave-tm1g krb5kdc[15478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:33.165249 15741 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:11:33.166935 15741 webserver.cc:460] Webserver started at http://127.11.60.2:37881/ using document root <none> and password file <none>
I20260811 03:11:33.167696 15741 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:33.167843 15741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:33.168179 15741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:33.171119 15741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "170a0d0cfd514a2d9590f8b9cd488183"
format_stamp: "Formatted at 2026-08-11 03:11:33 on dist-test-slave-tm1g"
server_key: "bf9336a7a0ddabc947e1637745d3c384"
server_key_iv: "e43054597b6cc81901b1693b15bc8603"
server_key_version: "encryptionkey@0"
I20260811 03:11:33.171932 15741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "170a0d0cfd514a2d9590f8b9cd488183"
format_stamp: "Formatted at 2026-08-11 03:11:33 on dist-test-slave-tm1g"
server_key: "bf9336a7a0ddabc947e1637745d3c384"
server_key_iv: "e43054597b6cc81901b1693b15bc8603"
server_key_version: "encryptionkey@0"
I20260811 03:11:33.177062 15741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:11:33.180491 15759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:33.181792 15741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:11:33.182000 15741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "170a0d0cfd514a2d9590f8b9cd488183"
format_stamp: "Formatted at 2026-08-11 03:11:33 on dist-test-slave-tm1g"
server_key: "bf9336a7a0ddabc947e1637745d3c384"
server_key_iv: "e43054597b6cc81901b1693b15bc8603"
server_key_version: "encryptionkey@0"
I20260811 03:11:33.182217 15741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:33.197530 15741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:33.201785 15741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:33.202091 15741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:33.203063 15741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:33.204433 15741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:33.204540 15741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:33.204696 15741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:33.204770 15741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:33.238924 15741 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:34039
I20260811 03:11:33.239051 15881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:34039 every 8 connection(s)
I20260811 03:11:33.240310 15741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:33.249539 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 15741
I20260811 03:11:33.249730 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:33.250233 11504 external_mini_cluster.cc:1428] Setting key 95b91c8d8af781e36dcb495d6ff9e9ae
Aug 11 03:11:33 dist-test-slave-tm1g krb5kdc[15478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:33.264719 15884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:33.242996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:49487 (local address 127.11.60.62:45965)
0811 03:11:33.243446 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:33.243451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:33.244501 (+ 1050us) server_negotiation.cc:407] Connection header received
0811 03:11:33.246614 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:33.246620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:33.246700 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:33.246839 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:33.248316 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.249110 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.251714 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.252400 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.252557 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:33.257848 (+ 5291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:33.257853 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:33.257857 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:33.257899 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:33.261004 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:33.261950 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:33.261958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:33.261963 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:33.262135 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:33.262677 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:33.262683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:33.262688 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:33.263100 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:33.263215 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:33.264235 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0811 03:11:33.264439 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":238,"threads_started":1}
I20260811 03:11:33.266535 15885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:33.243615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:45965 (local address 127.11.60.2:49487)
0811 03:11:33.244341 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:33.244384 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:33.246261 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:33.247076 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:33.247086 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:33.247586 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:33.248142 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:33.248158 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.249421 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.249428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:33.251548 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:33.251566 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.252863 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.252871 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:33.253173 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:33.254023 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:33.254072 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:33.257573 (+ 3501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:33.261164 (+ 3591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:33.261178 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:33.261200 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:33.261789 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:33.262252 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:33.262256 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:33.262258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:33.262541 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:33.263238 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:33.263250 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:33.263763 (+ 513us) client_negotiation.cc:769] Sending connection context
0811 03:11:33.264132 (+ 369us) client_negotiation.cc:240] Negotiation successful
0811 03:11:33.264786 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":296,"threads_started":1}
I20260811 03:11:33.268234 15882 heartbeater.cc:347] Connected to a master server at 127.11.60.62:45965
I20260811 03:11:33.268620 15882 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:33.269621 15882 heartbeater.cc:511] Master 127.11.60.62:45965 requested a full tablet report, sending...
I20260811 03:11:33.271515 15525 ts_manager.cc:194] Registered new tserver with Master: 170a0d0cfd514a2d9590f8b9cd488183 (127.11.60.2:34039)
I20260811 03:11:33.272460 15525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:49487
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:11:33.306221 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:45965
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:32773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:33.644248 15888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:33.644534 15888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:33.644629 15888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:33.652064 15888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:33.652171 15888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:33.652287 15888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:33.661845 15888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:32773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:45965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:33.663259 15888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:33.664649 15888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:33.678370 15893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:33.678352 15896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:33.678267 15894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:33.679358 15888 server_base.cc:1034] running on GCE node
I20260811 03:11:33.679962 15888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:33.680922 15888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:33.682155 15888 hybrid_clock.cc:648] HybridClock initialized: now 1786417893682113 us; error 68 us; skew 500 ppm
I20260811 03:11:33.682541 15888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:33 dist-test-slave-tm1g krb5kdc[15478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:33.685750 15888 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:11:33.687424 15888 webserver.cc:460] Webserver started at http://127.11.60.3:38763/ using document root <none> and password file <none>
I20260811 03:11:33.688127 15888 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:33.688277 15888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:33.688604 15888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:33.691532 15888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "c1aef3b3e89a45a1b11ddfe255c6e00f"
format_stamp: "Formatted at 2026-08-11 03:11:33 on dist-test-slave-tm1g"
server_key: "5197480ff320ec47aa8d980c7b654dbf"
server_key_iv: "59708d14a8e6769d40fc59af63645856"
server_key_version: "encryptionkey@0"
I20260811 03:11:33.692303 15888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "c1aef3b3e89a45a1b11ddfe255c6e00f"
format_stamp: "Formatted at 2026-08-11 03:11:33 on dist-test-slave-tm1g"
server_key: "5197480ff320ec47aa8d980c7b654dbf"
server_key_iv: "59708d14a8e6769d40fc59af63645856"
server_key_version: "encryptionkey@0"
I20260811 03:11:33.697392 15888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 03:11:33.700788 15905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:33.702001 15888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:11:33.702178 15888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "c1aef3b3e89a45a1b11ddfe255c6e00f"
format_stamp: "Formatted at 2026-08-11 03:11:33 on dist-test-slave-tm1g"
server_key: "5197480ff320ec47aa8d980c7b654dbf"
server_key_iv: "59708d14a8e6769d40fc59af63645856"
server_key_version: "encryptionkey@0"
I20260811 03:11:33.702462 15888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:33.716342 15888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:33.720583 15888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:33.720887 15888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:33.721814 15888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:33.723186 15888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:33.723289 15888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:33.723444 15888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:33.723526 15888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:33.748924 15735 heartbeater.cc:503] Master 127.11.60.62:45965 was elected leader, sending a full tablet report...
I20260811 03:11:33.761955 15888 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:34153
I20260811 03:11:33.761991 16025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:34153 every 8 connection(s)
I20260811 03:11:33.763808 15888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:33.768481 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 15888
I20260811 03:11:33.768695 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:33.769237 11504 external_mini_cluster.cc:1428] Setting key 7bbd6225d90ac66d80a7b226514f6795
Aug 11 03:11:33 dist-test-slave-tm1g krb5kdc[15478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:33.784214 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:33.766437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:49389 (local address 127.11.60.62:45965)
0811 03:11:33.766945 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:33.766950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:33.767878 (+ 928us) server_negotiation.cc:407] Connection header received
0811 03:11:33.770330 (+ 2452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:33.770336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:33.770450 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:33.770580 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:33.772170 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.773097 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.774685 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.775111 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.775240 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:33.778567 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:33.778572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:33.778575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:33.778605 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:33.780994 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:33.781695 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:33.781705 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:33.781707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:33.781823 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:33.782263 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:33.782268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:33.782273 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:33.782650 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:33.782807 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:33.783598 (+ 791us) server_negotiation.cc:299] Negotiation successful
0811 03:11:33.783816 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":271,"threads_started":1}
I20260811 03:11:33.785200 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:33.766912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:45965 (local address 127.11.60.3:49389)
0811 03:11:33.767723 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:33.767760 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:33.770052 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:33.770819 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:33.770832 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:33.771460 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:33.771997 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:33.772013 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.773258 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.773263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:33.774547 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:33.774559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.775340 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.775347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:33.775576 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:33.776217 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:33.776237 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:33.778367 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:33.781143 (+ 2776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:33.781151 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:33.781166 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:33.781567 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:33.781932 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:33.781937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:33.781941 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:33.782141 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:33.782797 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:33.782803 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:33.783203 (+ 400us) client_negotiation.cc:769] Sending connection context
0811 03:11:33.783473 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 03:11:33.783955 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":335,"threads_started":1}
I20260811 03:11:33.786571 16026 heartbeater.cc:347] Connected to a master server at 127.11.60.62:45965
I20260811 03:11:33.786902 16026 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:33.787787 16026 heartbeater.cc:511] Master 127.11.60.62:45965 requested a full tablet report, sending...
I20260811 03:11:33.789153 15525 ts_manager.cc:194] Registered new tserver with Master: c1aef3b3e89a45a1b11ddfe255c6e00f (127.11.60.3:34153)
I20260811 03:11:33.790014 15525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:49389
I20260811 03:11:33.796850 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 03:11:33 dist-test-slave-tm1g krb5kdc[15478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 03:11:33 dist-test-slave-tm1g krb5kdc[15478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:33.850693 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:33.835038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39262 (local address 127.11.60.1:34957)
0811 03:11:33.835801 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:33.835808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:33.835833 (+ 25us) server_negotiation.cc:407] Connection header received
0811 03:11:33.835895 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:33.835903 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:33.836123 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:33.836296 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:33.837749 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.838873 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.840110 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.840680 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.840826 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:33.844666 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:33.844677 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:33.844691 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:33.844722 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:33.847767 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:33.848431 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:33.848438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:33.848440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:33.848570 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:33.848946 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:33.848951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:33.848956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:33.849350 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:33.849712 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:33.849984 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 03:11:33.850258 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":381,"threads_started":1}
Aug 11 03:11:33 dist-test-slave-tm1g krb5kdc[15478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:33.869607 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:33.855612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.11.60.62:45965)
0811 03:11:33.855837 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:33.855843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:33.856047 (+ 204us) server_negotiation.cc:407] Connection header received
0811 03:11:33.856237 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:33.856241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:33.856307 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:33.856407 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:33.857601 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.858449 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.859687 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.860109 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.860243 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:33.864301 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:33.864306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:33.864310 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:33.864343 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:33.867048 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:33.867679 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:33.867687 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:33.867691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:33.867815 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:33.868172 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:33.868178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:33.868182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:33.868504 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:33.868761 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:33.869066 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 03:11:33.869292 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 03:11:33.870196 15525 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59148
I20260811 03:11:33.871423 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 15594
I20260811 03:11:33.884222 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:33.872301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.11.60.1:34957)
0811 03:11:33.872502 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:33.872506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:33.872878 (+ 372us) server_negotiation.cc:407] Connection header received
0811 03:11:33.873031 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:33.873036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:33.873100 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:33.873175 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:33.874448 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.875215 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.876423 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.876832 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.876929 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:33.879394 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:33.879399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:33.879402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:33.879431 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:33.881812 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:33.882547 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:33.882551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:33.882553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:33.882667 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:33.883056 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:33.883059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:33.883061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:33.883278 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:33.883444 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:33.883745 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 03:11:33.883897 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 03:11:33.884828 15728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:33.969365 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 15594
I20260811 03:11:33.989159 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 15741
Aug 11 03:11:33 dist-test-slave-tm1g krb5kdc[15478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:11:34.005188 16040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:33.989997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.11.60.2:34039)
0811 03:11:33.990626 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:33.990634 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:33.990660 (+ 26us) server_negotiation.cc:407] Connection header received
0811 03:11:33.990720 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:33.990726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:33.990946 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:33.991080 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:33.992347 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.993304 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.994593 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:33.995123 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:33.995226 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:33.999774 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:33.999786 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:33.999800 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:33.999831 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:34.002797 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:34.003448 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:34.003455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:34.003457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:34.003584 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:34.003903 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:34.003906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:34.003908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:34.004230 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:34.004452 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:34.004718 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 03:11:34.004904 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":343,"threads_started":1}
I20260811 03:11:34.006662 15876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:34.091240 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 15741
I20260811 03:11:34.111562 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 15888
Aug 11 03:11:34 dist-test-slave-tm1g krb5kdc[15478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:11:34.128262 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:34.112587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58316 (local address 127.11.60.3:34153)
0811 03:11:34.113227 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:34.113234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:34.113262 (+ 28us) server_negotiation.cc:407] Connection header received
0811 03:11:34.113320 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:34.113326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:34.113559 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:34.113685 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:34.115127 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:34.116142 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:34.117342 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:34.117891 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:34.117985 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:34.122208 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:34.122220 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:34.122235 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:34.122269 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:34.125126 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:34.125771 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:34.125778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:34.125780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:34.125917 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:34.126256 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:34.126260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:34.126263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:34.126795 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:34.127207 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:34.127562 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 03:11:34.127845 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":329,"threads_started":1}
I20260811 03:11:34.129606 16020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:34.213694 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 15888
I20260811 03:11:34.231395 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 15491
I20260811 03:11:34.232713 15560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:34.339077 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 15491
2026-08-11T03:11:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4045 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:11:34 dist-test-slave-tm1g krb5kdc[16048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:11:34 dist-test-slave-tm1g krb5kdc[16048](info): set up 2 sockets
Aug 11 03:11:34 dist-test-slave-tm1g krb5kdc[16048](info): commencing operation
krb5kdc: starting...
W20260811 03:11:35.490948 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:11:35 dist-test-slave-tm1g krb5kdc[16048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:11:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:11:35.722609 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:33529
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:33529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:36.062525 16062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:36.062892 16062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:36.063060 16062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:36.070176 16062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:36.070317 16062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:36.070451 16062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:36.070552 16062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:36.070644 16062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:36.080341 16062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:33529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:33529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:36.081668 16062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:36.083055 16062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:36.095162 16067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:36.095115 16070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:36.095185 16068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:36.096055 16062 server_base.cc:1034] running on GCE node
I20260811 03:11:36.096892 16062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:36.098289 16062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:36.099586 16062 hybrid_clock.cc:648] HybridClock initialized: now 1786417896099539 us; error 44 us; skew 500 ppm
I20260811 03:11:36.099936 16062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:36 dist-test-slave-tm1g krb5kdc[16048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417896, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:36.103125 16062 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:11:36.104272 16062 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:36.104358 16062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:36.104609 16062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:36.108173 16062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff"
format_stamp: "Formatted at 2026-08-11 03:11:36 on dist-test-slave-tm1g"
server_key: "83f4acb083aec64b4111b7c31e6330f3"
server_key_iv: "e9605f4fce854777d95f8dcc0c48ea48"
server_key_version: "encryptionkey@0"
I20260811 03:11:36.109217 16062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff"
format_stamp: "Formatted at 2026-08-11 03:11:36 on dist-test-slave-tm1g"
server_key: "83f4acb083aec64b4111b7c31e6330f3"
server_key_iv: "e9605f4fce854777d95f8dcc0c48ea48"
server_key_version: "encryptionkey@0"
I20260811 03:11:36.114570 16062 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 03:11:36.118158 16077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:36.119546 16062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:36.119694 16062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff"
format_stamp: "Formatted at 2026-08-11 03:11:36 on dist-test-slave-tm1g"
server_key: "83f4acb083aec64b4111b7c31e6330f3"
server_key_iv: "e9605f4fce854777d95f8dcc0c48ea48"
server_key_version: "encryptionkey@0"
I20260811 03:11:36.119875 16062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:36.130815 16062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:36.134922 16062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:36.135130 16062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:36.153420 16062 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:33529
I20260811 03:11:36.153440 16129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:33529 every 8 connection(s)
I20260811 03:11:36.154327 16062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:36.157943 16130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:36.164393 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 16062
I20260811 03:11:36.164633 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:36.165325 11504 external_mini_cluster.cc:1428] Setting key a9de869aa984ec616b3b9de934491ad9
I20260811 03:11:36.168757 16130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff: Bootstrap starting.
I20260811 03:11:36.171830 16130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:36.172886 16130 log.cc:824] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:36.175733 16130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff: No bootstrap required, opened a new log
Aug 11 03:11:36 dist-test-slave-tm1g krb5kdc[16048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:36.184095 16130 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33529 } }
I20260811 03:11:36.184489 16130 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:36.184625 16130 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ae9b0b428c4de7a98a4ff8cbdf62ff, State: Initialized, Role: FOLLOWER
I20260811 03:11:36.185308 16130 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33529 } }
I20260811 03:11:36.185586 16130 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:36.185751 16130 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:36.185964 16130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:36.187856 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:36.167339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43756 (local address 127.11.60.62:33529)
0811 03:11:36.168107 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:36.168118 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:36.168157 (+ 39us) server_negotiation.cc:407] Connection header received
0811 03:11:36.169865 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:36.169886 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:36.170255 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:36.170719 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:36.172292 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:36.173669 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:36.175050 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:36.175712 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:36.175910 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:36.180149 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:36.180157 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:36.180173 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:36.180208 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:36.183576 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:36.184272 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:36.184279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:36.184285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:36.184505 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:36.184981 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:36.184986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:36.184990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:36.185567 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:36.185884 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:36.186343 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 03:11:36.186790 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":338,"threads_started":1}
I20260811 03:11:36.188724 16130 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33529 } }
I20260811 03:11:36.189231 16130 leader_election.cc:302] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1ae9b0b428c4de7a98a4ff8cbdf62ff; no voters:
I20260811 03:11:36.189908 16130 leader_election.cc:288] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:36.190033 16135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:36.191185 16135 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [term 1 LEADER]: Becoming Leader. State: Replica: b1ae9b0b428c4de7a98a4ff8cbdf62ff, State: Running, Role: LEADER
I20260811 03:11:36.191771 16135 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33529 } }
I20260811 03:11:36.192579 16130 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:36.194778 16137 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33529 } } }
I20260811 03:11:36.195240 16137 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:36.196509 16138 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1ae9b0b428c4de7a98a4ff8cbdf62ff. Latest consensus state: current_term: 1 leader_uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ae9b0b428c4de7a98a4ff8cbdf62ff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 33529 } } }
I20260811 03:11:36.196950 16138 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:36.199523 16144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:36.205188 16144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:36.214730 16144 catalog_manager.cc:1347] Generated new cluster ID: f735a5d8636e403bb57e2f36dbf0ea5a
I20260811 03:11:36.214862 16144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:36.225807 16144 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:36.227234 16144 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:36.233558 16144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff: Generated new TSK 0
I20260811 03:11:36.234490 16144 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:11:36.297981 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33529
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:36.636745 16159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:36.637017 16159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:36.637209 16159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:36.644819 16159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:36.644929 16159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:36.645123 16159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:36.655159 16159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:36.656579 16159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:36.658015 16159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:36.671406 16165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:36.671522 16167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:36.671461 16164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:36.676041 16159 server_base.cc:1034] running on GCE node
I20260811 03:11:36.676723 16159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:36.677747 16159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:36.678977 16159 hybrid_clock.cc:648] HybridClock initialized: now 1786417896678936 us; error 53 us; skew 500 ppm
I20260811 03:11:36.679315 16159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:36 dist-test-slave-tm1g krb5kdc[16048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417896, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:36.682526 16159 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:11:36.683835 16159 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:36.683974 16159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:36.684303 16159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:36.687544 16159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "e2ff790a82c14b9cad42f9dd327c7083"
format_stamp: "Formatted at 2026-08-11 03:11:36 on dist-test-slave-tm1g"
server_key: "b3a8ccb2860738a70def3bac86586bcf"
server_key_iv: "1e5035a6d7b44b860fad2addb1faad42"
server_key_version: "encryptionkey@0"
I20260811 03:11:36.688383 16159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "e2ff790a82c14b9cad42f9dd327c7083"
format_stamp: "Formatted at 2026-08-11 03:11:36 on dist-test-slave-tm1g"
server_key: "b3a8ccb2860738a70def3bac86586bcf"
server_key_iv: "1e5035a6d7b44b860fad2addb1faad42"
server_key_version: "encryptionkey@0"
I20260811 03:11:36.693522 16159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:11:36.697420 16174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:36.698841 16159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:36.699038 16159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "e2ff790a82c14b9cad42f9dd327c7083"
format_stamp: "Formatted at 2026-08-11 03:11:36 on dist-test-slave-tm1g"
server_key: "b3a8ccb2860738a70def3bac86586bcf"
server_key_iv: "1e5035a6d7b44b860fad2addb1faad42"
server_key_version: "encryptionkey@0"
I20260811 03:11:36.699272 16159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:36.724187 16159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:36.728789 16159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:36.729086 16159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:36.730089 16159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:36.731629 16159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:36.731760 16159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:36.731894 16159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:36.731983 16159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:36.765407 16159 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:42087
I20260811 03:11:36.765453 16298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:42087 every 8 connection(s)
I20260811 03:11:36.766371 16159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:36.771668 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 16159
I20260811 03:11:36.771909 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:36.772567 11504 external_mini_cluster.cc:1428] Setting key 9982e698ac2d128d27c51186ac7241e5
Aug 11 03:11:36 dist-test-slave-tm1g krb5kdc[16048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417896, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:36.788183 16302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:36.768941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:59891 (local address 127.11.60.62:33529)
0811 03:11:36.769437 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:36.769444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:36.770463 (+ 1019us) server_negotiation.cc:407] Connection header received
0811 03:11:36.772388 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:36.772394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:36.772484 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:36.772613 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:36.774460 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:36.775559 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:36.777158 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:36.777855 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:36.778013 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:36.781638 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:36.781647 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:36.781655 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:36.781702 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:36.784599 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:36.785663 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:36.785667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:36.785669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:36.785798 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:36.786293 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:36.786296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:36.786298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:36.786637 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:36.786756 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:36.787733 (+ 977us) server_negotiation.cc:299] Negotiation successful
0811 03:11:36.787908 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":236,"threads_started":1}
I20260811 03:11:36.789808 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:36.769549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:33529 (local address 127.11.60.1:59891)
0811 03:11:36.770251 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:36.770342 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:36.772120 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:36.772832 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:36.772841 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:36.773498 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:36.774253 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:36.774279 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:36.775730 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:36.775734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:36.776992 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:36.777004 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:36.778075 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:36.778081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:36.778263 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:36.779033 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:36.779055 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:36.781407 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:36.784770 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:36.784780 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:36.784800 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:36.785526 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:36.785907 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:36.785915 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:36.785921 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:36.786179 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:36.786768 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:36.786779 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:36.787295 (+ 516us) client_negotiation.cc:769] Sending connection context
0811 03:11:36.787618 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 03:11:36.788234 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":290,"threads_started":1}
I20260811 03:11:36.791527 16299 heartbeater.cc:347] Connected to a master server at 127.11.60.62:33529
I20260811 03:11:36.791847 16299 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:36.792623 16299 heartbeater.cc:511] Master 127.11.60.62:33529 requested a full tablet report, sending...
I20260811 03:11:36.795150 16094 ts_manager.cc:194] Registered new tserver with Master: e2ff790a82c14b9cad42f9dd327c7083 (127.11.60.1:42087)
I20260811 03:11:36.796613 16094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:59891
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:11:36.829911 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33529
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:37.169386 16306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:37.169668 16306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:37.169842 16306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:37.177235 16306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:37.177348 16306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:37.177561 16306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:37.187862 16306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:37.189342 16306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:37.190793 16306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:37.204806 16311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:37.204839 16314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:37.204910 16312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:37.205905 16306 server_base.cc:1034] running on GCE node
I20260811 03:11:37.206645 16306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:37.207567 16306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:37.208775 16306 hybrid_clock.cc:648] HybridClock initialized: now 1786417897208734 us; error 34 us; skew 500 ppm
I20260811 03:11:37.209066 16306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:37 dist-test-slave-tm1g krb5kdc[16048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417897, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:37.217157 16306 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:11:37.218513 16306 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:37.218654 16306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:37.218994 16306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:37.222018 16306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "105dbab9b43642cfadb832010566f205"
format_stamp: "Formatted at 2026-08-11 03:11:37 on dist-test-slave-tm1g"
server_key: "054b5c234d728629b0a8785223926a05"
server_key_iv: "aaf0726984bc16f4b6271200df23968c"
server_key_version: "encryptionkey@0"
I20260811 03:11:37.222965 16306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "105dbab9b43642cfadb832010566f205"
format_stamp: "Formatted at 2026-08-11 03:11:37 on dist-test-slave-tm1g"
server_key: "054b5c234d728629b0a8785223926a05"
server_key_iv: "aaf0726984bc16f4b6271200df23968c"
server_key_version: "encryptionkey@0"
I20260811 03:11:37.228116 16306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:11:37.231500 16320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:37.232802 16306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:11:37.232987 16306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "105dbab9b43642cfadb832010566f205"
format_stamp: "Formatted at 2026-08-11 03:11:37 on dist-test-slave-tm1g"
server_key: "054b5c234d728629b0a8785223926a05"
server_key_iv: "aaf0726984bc16f4b6271200df23968c"
server_key_version: "encryptionkey@0"
I20260811 03:11:37.233238 16306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:37.253558 16306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:37.257992 16306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:37.258283 16306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:37.259320 16306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:37.260720 16306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:37.260846 16306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:37.260987 16306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:37.261080 16306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:37.296211 16306 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:40657
I20260811 03:11:37.296259 16443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:40657 every 8 connection(s)
I20260811 03:11:37.297189 16306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:37.303623 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 16306
I20260811 03:11:37.303831 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:37.304332 11504 external_mini_cluster.cc:1428] Setting key 2f6176096758ac039a82527809b8402f
Aug 11 03:11:37 dist-test-slave-tm1g krb5kdc[16048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417897, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:37.319469 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.299841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:60079 (local address 127.11.60.62:33529)
0811 03:11:37.300304 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:37.300308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:37.301254 (+ 946us) server_negotiation.cc:407] Connection header received
0811 03:11:37.303180 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:37.303210 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:37.303304 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:37.303426 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:37.305051 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.306096 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.308559 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.308997 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.309101 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.313279 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:37.313285 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:37.313288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:37.313320 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:37.316141 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:37.316868 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:37.316876 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:37.316883 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:37.317039 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:37.317426 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:37.317434 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:37.317440 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:37.317828 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:37.317977 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:37.318717 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 03:11:37.318968 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":253,"threads_started":1}
I20260811 03:11:37.320096 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.300398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:33529 (local address 127.11.60.2:60079)
0811 03:11:37.301094 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:37.301138 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:37.302904 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:37.303656 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:37.303664 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:37.304259 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:37.304869 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:37.304886 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.306268 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.306272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:37.308393 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:37.308413 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.309226 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.309236 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:37.309538 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.310326 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:37.310364 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:37.313041 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:37.316289 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:37.316296 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:37.316309 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:37.316733 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:37.317144 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:37.317148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:37.317150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:37.317312 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:37.317953 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:37.317958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:37.318259 (+ 301us) client_negotiation.cc:769] Sending connection context
0811 03:11:37.318565 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 03:11:37.318958 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":294,"threads_started":1}
I20260811 03:11:37.321717 16444 heartbeater.cc:347] Connected to a master server at 127.11.60.62:33529
I20260811 03:11:37.322074 16444 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:37.322923 16444 heartbeater.cc:511] Master 127.11.60.62:33529 requested a full tablet report, sending...
I20260811 03:11:37.324762 16094 ts_manager.cc:194] Registered new tserver with Master: 105dbab9b43642cfadb832010566f205 (127.11.60.2:40657)
I20260811 03:11:37.326040 16094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:60079
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:11:37.361478 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33529
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:37.703162 16450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:37.703469 16450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:37.703572 16450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:37.711112 16450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:37.711220 16450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:37.711347 16450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:37.721215 16450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:33529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:37.722568 16450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:37.723979 16450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:37.737100 16455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:37.737149 16456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:37.737177 16458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:37.738140 16450 server_base.cc:1034] running on GCE node
I20260811 03:11:37.738881 16450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:37.742641 16450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:37.743911 16450 hybrid_clock.cc:648] HybridClock initialized: now 1786417897743845 us; error 54 us; skew 500 ppm
I20260811 03:11:37.744323 16450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:37 dist-test-slave-tm1g krb5kdc[16048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417897, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:37.748239 16450 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:11:37.749457 16450 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:37.749596 16450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:37.749954 16450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:37.753046 16450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "5e8bd948c45e420cbfcf7eebd913533d"
format_stamp: "Formatted at 2026-08-11 03:11:37 on dist-test-slave-tm1g"
server_key: "760df456f789ec6b4f9b3a703ded7ed2"
server_key_iv: "9c592931deff616eec3158bba6569294"
server_key_version: "encryptionkey@0"
I20260811 03:11:37.753896 16450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "5e8bd948c45e420cbfcf7eebd913533d"
format_stamp: "Formatted at 2026-08-11 03:11:37 on dist-test-slave-tm1g"
server_key: "760df456f789ec6b4f9b3a703ded7ed2"
server_key_iv: "9c592931deff616eec3158bba6569294"
server_key_version: "encryptionkey@0"
I20260811 03:11:37.759227 16450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 03:11:37.763120 16464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:37.764740 16450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:11:37.764925 16450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "5e8bd948c45e420cbfcf7eebd913533d"
format_stamp: "Formatted at 2026-08-11 03:11:37 on dist-test-slave-tm1g"
server_key: "760df456f789ec6b4f9b3a703ded7ed2"
server_key_iv: "9c592931deff616eec3158bba6569294"
server_key_version: "encryptionkey@0"
I20260811 03:11:37.765182 16450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:37.779258 16450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:37.783784 16450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:37.784065 16450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:37.785141 16450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:37.786672 16450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:37.786763 16450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:37.786851 16450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:37.786906 16450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:37.799870 16299 heartbeater.cc:503] Master 127.11.60.62:33529 was elected leader, sending a full tablet report...
I20260811 03:11:37.821216 16450 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:36863
I20260811 03:11:37.821269 16587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:36863 every 8 connection(s)
I20260811 03:11:37.822156 16450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:37.825340 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 16450
I20260811 03:11:37.825549 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:37.826066 11504 external_mini_cluster.cc:1428] Setting key 5c27de7cdda3c64165b1105a17c754f8
Aug 11 03:11:37 dist-test-slave-tm1g krb5kdc[16048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417897, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:37.843639 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.824964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:59245 (local address 127.11.60.62:33529)
0811 03:11:37.825601 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:37.825610 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:37.826858 (+ 1248us) server_negotiation.cc:407] Connection header received
0811 03:11:37.829005 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:37.829010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:37.829120 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:37.829246 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:37.830854 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.831693 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.833174 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.833622 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.833720 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.837228 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:37.837233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:37.837236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:37.837296 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:37.840254 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:37.841153 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:37.841159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:37.841164 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:37.841328 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:37.841870 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:37.841888 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:37.841891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:37.842246 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:37.842498 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:37.843151 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 03:11:37.843331 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":305,"threads_started":1}
I20260811 03:11:37.844599 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.825614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:33529 (local address 127.11.60.3:59245)
0811 03:11:37.826451 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:37.826523 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:37.828743 (+ 2220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:37.829505 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:37.829518 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:37.830048 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:37.830680 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:37.830697 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.831815 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.831819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:37.833041 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:37.833052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.833809 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.833813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:37.833993 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.834698 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:37.834726 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:37.837031 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:37.840390 (+ 3359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:37.840399 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:37.840415 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:37.840976 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:37.841467 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:37.841471 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:37.841474 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:37.841744 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:37.842482 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:37.842488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:37.842807 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 03:11:37.843053 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 03:11:37.843434 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":315,"threads_started":1}
I20260811 03:11:37.846069 16588 heartbeater.cc:347] Connected to a master server at 127.11.60.62:33529
I20260811 03:11:37.846426 16588 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:37.847091 16588 heartbeater.cc:511] Master 127.11.60.62:33529 requested a full tablet report, sending...
I20260811 03:11:37.848645 16094 ts_manager.cc:194] Registered new tserver with Master: 5e8bd948c45e420cbfcf7eebd913533d (127.11.60.3:36863)
I20260811 03:11:37.849515 16094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:59245
I20260811 03:11:37.854060 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:11:37.871805 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.858660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.11.60.62:33529)
0811 03:11:37.858821 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:37.858826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:37.859353 (+ 527us) server_negotiation.cc:407] Connection header received
0811 03:11:37.859457 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:37.859460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:37.859523 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:37.859604 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:37.861152 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.861909 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.863311 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.863739 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.863845 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.866201 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:37.866207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:37.866211 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:37.866244 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:37.868979 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:37.869831 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:37.869836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:37.869844 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:37.869970 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:37.870363 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:37.870370 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:37.870378 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:37.870699 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:37.870897 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:37.871233 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 03:11:37.871429 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 03:11:37.876034 16094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:11:37.883030 16094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:37.909999 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.896409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.11.60.1:42087)
0811 03:11:37.900121 (+ 3712us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:37.900131 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:37.900872 (+ 741us) server_negotiation.cc:407] Connection header received
0811 03:11:37.900992 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:37.901001 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:37.901306 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:37.901514 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:37.902822 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.904451 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.907404 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.908723 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.908861 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.909116 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:37.909442 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 03:11:37.909635 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3488,"thread_start_us":3803,"threads_started":1}
I20260811 03:11:37.910080 16602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.896108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:42087 (local address 127.0.0.1:60070)
0811 03:11:37.900365 (+ 4257us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:37.900415 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:37.900639 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:37.901710 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:37.901720 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:37.901766 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:37.902231 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:37.902620 (+ 389us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.904611 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.904617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:37.907239 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:37.907252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.908929 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.908936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:37.909154 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.909211 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 03:11:37.909343 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 03:11:37.909547 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1348,"mutex_wait_us":3,"thread_start_us":4474,"threads_started":1}
I20260811 03:11:37.910568 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.896248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:40657 (local address 127.0.0.1:58686)
0811 03:11:37.900873 (+ 4625us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:37.900891 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:37.900990 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:37.903233 (+ 2243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:37.903238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:37.903262 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:37.903753 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:37.903766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.905417 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.905425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:37.907775 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:37.907791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.909603 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.909610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:37.909812 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.909863 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 03:11:37.909946 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 03:11:37.910110 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4506,"mutex_wait_us":65,"thread_start_us":1003,"threads_started":1}
I20260811 03:11:37.911149 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.898864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.11.60.3:36863)
0811 03:11:37.901581 (+ 2717us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:37.901592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:37.902562 (+ 970us) server_negotiation.cc:407] Connection header received
0811 03:11:37.902680 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:37.902687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:37.902898 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:37.903159 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:37.904208 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.905775 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.908251 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.909479 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.909610 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.909832 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:37.910669 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 03:11:37.910871 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2530,"thread_start_us":2349,"threads_started":1}
I20260811 03:11:37.911659 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.896891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58686 (local address 127.11.60.2:40657)
0811 03:11:37.902627 (+ 5736us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:37.902638 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:37.902666 (+ 28us) server_negotiation.cc:407] Connection header received
0811 03:11:37.902760 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:37.902767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:37.903012 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:37.903810 (+ 798us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:37.904037 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.905254 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.907953 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.909368 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.910958 (+ 1590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.911161 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:37.911234 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 03:11:37.911406 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5575,"thread_start_us":5451,"threads_started":1}
I20260811 03:11:37.911723 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:37.898165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:36863 (local address 127.0.0.1:47130)
0811 03:11:37.902286 (+ 4121us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:37.902323 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:37.902441 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:37.903518 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:37.903524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:37.903553 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:37.904017 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:37.904026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.905941 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.905946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:37.908097 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:37.908108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:37.910013 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:37.910021 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:37.910208 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:37.910241 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 03:11:37.910402 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 03:11:37.910538 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3930,"mutex_wait_us":26,"thread_start_us":3759,"threads_started":1}
I20260811 03:11:37.915009 16521 tablet_service.cc:1467] Processing CreateTablet for tablet 1bcfd533b3ab481abcf372f4190af815 (DEFAULT_TABLE table=test-table [id=f5dcf908af994208880de4ebe39086d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:37.915151 16373 tablet_service.cc:1467] Processing CreateTablet for tablet 1bcfd533b3ab481abcf372f4190af815 (DEFAULT_TABLE table=test-table [id=f5dcf908af994208880de4ebe39086d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:37.915151 16229 tablet_service.cc:1467] Processing CreateTablet for tablet 1bcfd533b3ab481abcf372f4190af815 (DEFAULT_TABLE table=test-table [id=f5dcf908af994208880de4ebe39086d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:37.916343 16521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bcfd533b3ab481abcf372f4190af815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:37.916528 16229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bcfd533b3ab481abcf372f4190af815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:37.916548 16373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bcfd533b3ab481abcf372f4190af815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:37.928098 16607 tablet_bootstrap.cc:492] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: Bootstrap starting.
I20260811 03:11:37.931978 16607 tablet_bootstrap.cc:654] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:37.932559 16609 tablet_bootstrap.cc:492] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: Bootstrap starting.
I20260811 03:11:37.933532 16607 log.cc:824] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:37.934084 16608 tablet_bootstrap.cc:492] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: Bootstrap starting.
I20260811 03:11:37.936131 16609 tablet_bootstrap.cc:654] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:37.937492 16609 log.cc:824] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:37.938181 16608 tablet_bootstrap.cc:654] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:37.939492 16607 tablet_bootstrap.cc:492] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: No bootstrap required, opened a new log
I20260811 03:11:37.939476 16608 log.cc:824] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:37.939781 16607 ts_tablet_manager.cc:1397] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260811 03:11:37.940615 16609 tablet_bootstrap.cc:492] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: No bootstrap required, opened a new log
I20260811 03:11:37.940872 16609 ts_tablet_manager.cc:1397] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 03:11:37.944970 16608 tablet_bootstrap.cc:492] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: No bootstrap required, opened a new log
I20260811 03:11:37.945242 16608 ts_tablet_manager.cc:1397] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260811 03:11:37.948223 16609 raft_consensus.cc:348] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } }
I20260811 03:11:37.948594 16609 raft_consensus.cc:374] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:37.948700 16609 raft_consensus.cc:729] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2ff790a82c14b9cad42f9dd327c7083, State: Initialized, Role: FOLLOWER
I20260811 03:11:37.949322 16609 consensus_queue.cc:260] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } }
I20260811 03:11:37.951083 16607 raft_consensus.cc:348] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } }
I20260811 03:11:37.951524 16607 raft_consensus.cc:374] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:37.951627 16607 raft_consensus.cc:729] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e8bd948c45e420cbfcf7eebd913533d, State: Initialized, Role: FOLLOWER
I20260811 03:11:37.952661 16609 ts_tablet_manager.cc:1428] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260811 03:11:37.953153 16607 consensus_queue.cc:260] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } }
I20260811 03:11:37.955053 16588 heartbeater.cc:503] Master 127.11.60.62:33529 was elected leader, sending a full tablet report...
I20260811 03:11:37.955822 16607 ts_tablet_manager.cc:1428] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: Time spent starting tablet: real 0.016s user 0.010s sys 0.007s
I20260811 03:11:37.955710 16608 raft_consensus.cc:348] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } }
I20260811 03:11:37.956148 16608 raft_consensus.cc:374] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:37.956261 16608 raft_consensus.cc:729] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 105dbab9b43642cfadb832010566f205, State: Initialized, Role: FOLLOWER
I20260811 03:11:37.956902 16608 consensus_queue.cc:260] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } }
I20260811 03:11:37.958632 16444 heartbeater.cc:503] Master 127.11.60.62:33529 was elected leader, sending a full tablet report...
I20260811 03:11:37.958858 16608 ts_tablet_manager.cc:1428] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
W20260811 03:11:38.018975 16300 tablet.cc:2369] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:11:38.049345 16445 tablet.cc:2369] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:38.063489 16616 raft_consensus.cc:482] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:38.063773 16616 raft_consensus.cc:504] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } }
I20260811 03:11:38.065675 16616 leader_election.cc:288] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2ff790a82c14b9cad42f9dd327c7083 (127.11.60.1:42087), 5e8bd948c45e420cbfcf7eebd913533d (127.11.60.3:36863)
I20260811 03:11:38.067895 16615 raft_consensus.cc:482] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:38.068225 16615 raft_consensus.cc:504] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } }
I20260811 03:11:38.070345 16615 leader_election.cc:288] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2ff790a82c14b9cad42f9dd327c7083 (127.11.60.1:42087), 105dbab9b43642cfadb832010566f205 (127.11.60.2:40657)
W20260811 03:11:38.073734 16589 tablet.cc:2369] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:38.076673 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.066525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:53211 (local address 127.11.60.1:42087)
0811 03:11:38.066819 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:38.066829 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:38.067602 (+ 773us) server_negotiation.cc:407] Connection header received
0811 03:11:38.067861 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:38.067866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:38.067933 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:38.067983 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:38.068879 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.069944 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.073221 (+ 3277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.074994 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.075154 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.075306 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:38.076160 (+ 854us) server_negotiation.cc:299] Negotiation successful
0811 03:11:38.076372 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 03:11:38.077541 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.066073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:42087 (local address 127.11.60.2:53211)
0811 03:11:38.067454 (+ 1381us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:38.067477 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:38.067663 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:38.068267 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:38.068270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:38.068292 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:38.068682 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:38.068694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.070079 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.070084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:38.072953 (+ 2869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:38.072976 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.075625 (+ 2649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.075634 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:38.075895 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.075947 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 03:11:38.076869 (+ 922us) client_negotiation.cc:240] Negotiation successful
0811 03:11:38.077032 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"mutex_wait_us":4,"thread_start_us":1125,"threads_started":1}
I20260811 03:11:38.078231 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.066344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:58439 (local address 127.11.60.3:36863)
0811 03:11:38.066723 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:38.066733 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:38.067680 (+ 947us) server_negotiation.cc:407] Connection header received
0811 03:11:38.068284 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:38.068289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:38.068424 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:38.068547 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:38.069281 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.070457 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.075043 (+ 4586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.076347 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.076494 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.076600 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:38.077822 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0811 03:11:38.077971 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":151}
I20260811 03:11:38.078996 16249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcfd533b3ab481abcf372f4190af815" candidate_uuid: "105dbab9b43642cfadb832010566f205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2ff790a82c14b9cad42f9dd327c7083" is_pre_election: true
I20260811 03:11:38.079478 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.071449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:56181 (local address 127.11.60.2:40657)
0811 03:11:38.071639 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:38.071652 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:38.071769 (+ 117us) server_negotiation.cc:407] Connection header received
0811 03:11:38.072019 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:38.072026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:38.072102 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:38.072163 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:38.072898 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.073973 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.076960 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.078273 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.078421 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.078515 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:38.079077 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 03:11:38.079229 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 03:11:38.079617 16249 raft_consensus.cc:2454] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 105dbab9b43642cfadb832010566f205 in term 0.
I20260811 03:11:38.080207 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.070993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:40657 (local address 127.11.60.3:56181)
0811 03:11:38.071667 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:38.071691 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:38.071866 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:38.072350 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:38.072355 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:38.072385 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:38.072713 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:38.072724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.074178 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.074184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:38.076790 (+ 2606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:38.076804 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.078695 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.078703 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:38.078887 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.078917 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 03:11:38.079646 (+ 729us) client_negotiation.cc:240] Negotiation successful
0811 03:11:38.079802 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":1,"thread_start_us":656,"threads_started":1}
I20260811 03:11:38.080688 16322 leader_election.cc:302] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 105dbab9b43642cfadb832010566f205, e2ff790a82c14b9cad42f9dd327c7083; no voters:
I20260811 03:11:38.080838 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.066107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:36863 (local address 127.11.60.2:58439)
0811 03:11:38.067591 (+ 1484us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:38.067607 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:38.067708 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:38.068578 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:38.068582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:38.068695 (+ 113us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:38.069105 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:38.069118 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.070618 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.070623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:38.074848 (+ 4225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:38.074864 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.077411 (+ 2547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.077418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:38.077629 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.077665 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 03:11:38.080130 (+ 2465us) client_negotiation.cc:240] Negotiation successful
0811 03:11:38.080426 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1346,"mutex_wait_us":2,"thread_start_us":758,"threads_started":1}
I20260811 03:11:38.081453 16616 raft_consensus.cc:2781] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:38.081625 16616 raft_consensus.cc:482] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:38.081766 16616 raft_consensus.cc:3037] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:38.081750 16541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcfd533b3ab481abcf372f4190af815" candidate_uuid: "105dbab9b43642cfadb832010566f205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e8bd948c45e420cbfcf7eebd913533d" is_pre_election: true
I20260811 03:11:38.081768 16394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcfd533b3ab481abcf372f4190af815" candidate_uuid: "5e8bd948c45e420cbfcf7eebd913533d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "105dbab9b43642cfadb832010566f205" is_pre_election: true
I20260811 03:11:38.082237 16541 raft_consensus.cc:2454] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 105dbab9b43642cfadb832010566f205 in term 0.
I20260811 03:11:38.084384 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.070694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:42087 (local address 127.11.60.3:43469)
0811 03:11:38.071368 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:38.071387 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:38.071519 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:38.077088 (+ 5569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:38.077093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:38.077123 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:38.077343 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:38.077354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.078997 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.079014 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:38.080679 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:38.080692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.083611 (+ 2919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.083617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:38.083840 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.083862 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 03:11:38.083969 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 03:11:38.084115 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544}
I20260811 03:11:38.084444 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.070946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:43469 (local address 127.11.60.1:42087)
0811 03:11:38.076780 (+ 5834us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:38.076786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:38.076807 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:11:38.076866 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:38.076872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:38.076967 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:38.077063 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:38.077553 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.078860 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.080839 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.083297 (+ 2458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.083604 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.083724 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:38.083995 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 03:11:38.084129 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5690,"thread_start_us":5461,"threads_started":1}
I20260811 03:11:38.084421 16616 raft_consensus.cc:504] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } }
I20260811 03:11:38.085130 16394 raft_consensus.cc:2379] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e8bd948c45e420cbfcf7eebd913533d in current term 1: Already voted for candidate 105dbab9b43642cfadb832010566f205 in this term.
I20260811 03:11:38.085244 16616 leader_election.cc:288] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [CANDIDATE]: Term 1 election: Requested vote from peers e2ff790a82c14b9cad42f9dd327c7083 (127.11.60.1:42087), 5e8bd948c45e420cbfcf7eebd913533d (127.11.60.3:36863)
I20260811 03:11:38.085224 16249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcfd533b3ab481abcf372f4190af815" candidate_uuid: "5e8bd948c45e420cbfcf7eebd913533d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2ff790a82c14b9cad42f9dd327c7083" is_pre_election: true
I20260811 03:11:38.085477 16249 raft_consensus.cc:2454] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e8bd948c45e420cbfcf7eebd913533d in term 0.
I20260811 03:11:38.085809 16541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcfd533b3ab481abcf372f4190af815" candidate_uuid: "105dbab9b43642cfadb832010566f205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e8bd948c45e420cbfcf7eebd913533d"
I20260811 03:11:38.086021 16541 raft_consensus.cc:3037] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:38.085975 16466 leader_election.cc:302] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e8bd948c45e420cbfcf7eebd913533d, e2ff790a82c14b9cad42f9dd327c7083; no voters:
I20260811 03:11:38.086979 16248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcfd533b3ab481abcf372f4190af815" candidate_uuid: "105dbab9b43642cfadb832010566f205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2ff790a82c14b9cad42f9dd327c7083"
I20260811 03:11:38.087178 16248 raft_consensus.cc:3037] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:38.088840 16541 raft_consensus.cc:2454] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 105dbab9b43642cfadb832010566f205 in term 1.
I20260811 03:11:38.089037 16615 raft_consensus.cc:2741] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 03:11:38.089306 16324 leader_election.cc:302] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 105dbab9b43642cfadb832010566f205, 5e8bd948c45e420cbfcf7eebd913533d; no voters:
I20260811 03:11:38.089634 16616 raft_consensus.cc:2781] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:38.089751 16248 raft_consensus.cc:2454] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 105dbab9b43642cfadb832010566f205 in term 1.
I20260811 03:11:38.090188 16616 raft_consensus.cc:686] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 1 LEADER]: Becoming Leader. State: Replica: 105dbab9b43642cfadb832010566f205, State: Running, Role: LEADER
I20260811 03:11:38.090780 16616 consensus_queue.cc:237] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } }
I20260811 03:11:38.095124 16093 catalog_manager.cc:5495] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 reported cstate change: term changed from 0 to 1, leader changed from <none> to 105dbab9b43642cfadb832010566f205 (127.11.60.2). New cstate: current_term: 1 leader_uuid: "105dbab9b43642cfadb832010566f205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "105dbab9b43642cfadb832010566f205" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260811 03:11:38.150969 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.145476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.11.60.2:40657)
0811 03:11:38.145690 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:38.145697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:38.145764 (+ 67us) server_negotiation.cc:407] Connection header received
0811 03:11:38.146018 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:38.146023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:38.146120 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:38.146272 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:38.146683 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.147447 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.148778 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.149180 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.149312 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.149807 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:38.150381 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:38.150541 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:38.150595 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 03:11:38.150676 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 03:11:38.160306 16541 raft_consensus.cc:1261] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 1 FOLLOWER]: Refusing update from remote peer 105dbab9b43642cfadb832010566f205: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:38.160393 16248 raft_consensus.cc:1261] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [term 1 FOLLOWER]: Refusing update from remote peer 105dbab9b43642cfadb832010566f205: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:38.161275 16616 consensus_queue.cc:1035] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e8bd948c45e420cbfcf7eebd913533d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:38.161621 16621 consensus_queue.cc:1035] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2ff790a82c14b9cad42f9dd327c7083" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 42087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:38.178880 16624 mvcc.cc:205] Tried to move back new op lower bound from 7317167710853550080 to 7317167710584061952. Current Snapshot: MvccSnapshot[applied={T|T < 7317167710853550080}]
I20260811 03:11:38.192879 16094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:38.193207 16094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:38.196951 16094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1ae9b0b428c4de7a98a4ff8cbdf62ff: Sending DeleteTablet for 3 replicas of tablet 1bcfd533b3ab481abcf372f4190af815
I20260811 03:11:38.198249 16229 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bcfd533b3ab481abcf372f4190af815 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:38 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:60070
I20260811 03:11:38.198480 16521 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bcfd533b3ab481abcf372f4190af815 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:38 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:47130
I20260811 03:11:38.199213 16373 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bcfd533b3ab481abcf372f4190af815 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:38 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:58686
I20260811 03:11:38.199339 16636 tablet_replica.cc:330] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: stopping tablet replica
I20260811 03:11:38.199975 16636 raft_consensus.cc:2229] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:38.200711 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 16159
I20260811 03:11:38.201036 16636 raft_consensus.cc:2258] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:38.206004 16636 ts_tablet_manager.cc:1905] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:38.209043 16637 tablet_replica.cc:330] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: stopping tablet replica
I20260811 03:11:38.209261 16635 tablet_replica.cc:330] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: stopping tablet replica
I20260811 03:11:38.209486 16637 raft_consensus.cc:2229] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:11:38.209632 16635 raft_consensus.cc:2229] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:38.210139 16635 raft_consensus.cc:2258] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:38.210139 16637 raft_consensus.cc:2258] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:38.212313 16635 ts_tablet_manager.cc:1905] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:38.212317 16637 ts_tablet_manager.cc:1905] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:38.213445 16636 ts_tablet_manager.cc:1918] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:38.213613 16636 log.cc:1196] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/1bcfd533b3ab481abcf372f4190af815
Aug 11 03:11:38 dist-test-slave-tm1g krb5kdc[16048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:38.214176 16636 ts_tablet_manager.cc:1939] T 1bcfd533b3ab481abcf372f4190af815 P 5e8bd948c45e420cbfcf7eebd913533d: Deleting consensus metadata
I20260811 03:11:38.219761 16637 ts_tablet_manager.cc:1918] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:38.219754 16635 ts_tablet_manager.cc:1918] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:38.219935 16635 log.cc:1196] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/1bcfd533b3ab481abcf372f4190af815
I20260811 03:11:38.219935 16637 log.cc:1196] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/1bcfd533b3ab481abcf372f4190af815
I20260811 03:11:38.220515 16635 ts_tablet_manager.cc:1939] T 1bcfd533b3ab481abcf372f4190af815 P e2ff790a82c14b9cad42f9dd327c7083: Deleting consensus metadata
I20260811 03:11:38.220535 16637 ts_tablet_manager.cc:1939] T 1bcfd533b3ab481abcf372f4190af815 P 105dbab9b43642cfadb832010566f205: Deleting consensus metadata
I20260811 03:11:38.221776 16081 catalog_manager.cc:4841] TS 5e8bd948c45e420cbfcf7eebd913533d (127.11.60.3:36863): tablet 1bcfd533b3ab481abcf372f4190af815 (table test-table [id=f5dcf908af994208880de4ebe39086d1]) successfully deleted
I20260811 03:11:38.222188 16079 catalog_manager.cc:4841] TS e2ff790a82c14b9cad42f9dd327c7083 (127.11.60.1:42087): tablet 1bcfd533b3ab481abcf372f4190af815 (table test-table [id=f5dcf908af994208880de4ebe39086d1]) successfully deleted
I20260811 03:11:38.222615 16081 catalog_manager.cc:4841] TS 105dbab9b43642cfadb832010566f205 (127.11.60.2:40657): tablet 1bcfd533b3ab481abcf372f4190af815 (table test-table [id=f5dcf908af994208880de4ebe39086d1]) successfully deleted
I20260811 03:11:38.223459 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.202561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.11.60.1:42087)
0811 03:11:38.202754 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:38.202760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:38.202778 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:11:38.202851 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:38.202857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:38.202948 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:38.203051 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:38.205047 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.206200 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.208105 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.208730 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.208828 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.215051 (+ 6223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:38.215069 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:38.215086 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:38.215131 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:38.219293 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:38.220471 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:38.220480 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:38.220483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:38.220765 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:38.221238 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:38.221244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:38.221250 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:38.221774 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:38.222166 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:38.222800 (+ 634us) server_negotiation.cc:299] Negotiation successful
0811 03:11:38.223057 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 03:11:38.224696 16293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:38.377156 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 16159
I20260811 03:11:38.397914 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 16306
Aug 11 03:11:38 dist-test-slave-tm1g krb5kdc[16048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:11:38.416610 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.398932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37916 (local address 127.11.60.2:40657)
0811 03:11:38.399140 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:38.399148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:38.399178 (+ 30us) server_negotiation.cc:407] Connection header received
0811 03:11:38.399252 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:38.399257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:38.399417 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:38.399541 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:38.400982 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.402177 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.403655 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.404372 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.404512 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.409071 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:38.409097 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:38.409119 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:38.409172 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:38.413378 (+ 4206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:38.414051 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:38.414060 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:38.414063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:38.414228 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:38.414622 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:38.414628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:38.414631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:38.415133 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:38.415427 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:38.415697 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 03:11:38.416210 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:11:38.417554 16439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:38.534101 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 16306
I20260811 03:11:38.556221 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 16450
Aug 11 03:11:38 dist-test-slave-tm1g krb5kdc[16048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:11:38.572407 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:38.557086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46634 (local address 127.11.60.3:36863)
0811 03:11:38.557289 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:38.557298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:38.557321 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:11:38.557558 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:38.557562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:38.557631 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:38.557741 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:38.559184 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.559941 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.561240 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:38.561653 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:38.561788 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:38.566115 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:38.566131 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:38.566149 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:38.566183 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:38.569104 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:38.569852 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:38.569861 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:38.569863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:38.570003 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:38.570715 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:38.570720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:38.570725 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:38.571150 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:38.571486 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:38.571826 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 03:11:38.572057 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:11:38.573103 16583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:38.665247 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 16450
I20260811 03:11:38.687911 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 16062
I20260811 03:11:38.689159 16125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:38.803354 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 16062
2026-08-11T03:11:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4467 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T03:11:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:38Z Disabled control of system clock
I20260811 03:11:38.882061 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:36431
--webserver_interface=127.11.60.62
--webserver_port=0
--builtin_ntp_servers=127.11.60.20:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:36431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 03:11:39.216149 16648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:39.216473 16648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:39.216629 16648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:39.223928 16648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:39.224035 16648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:39.224138 16648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:39.224255 16648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:39.224309 16648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:39.233629 16648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:36431
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:36431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:39.235028 16648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:39.236899 16648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:39.249102 16656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:39.249222 16654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:39.249099 16658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:39.249627 16648 server_base.cc:1034] running on GCE node
I20260811 03:11:39.250629 16648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:39.252043 16648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:39.253279 16648 hybrid_clock.cc:648] HybridClock initialized: now 1786417899253222 us; error 52 us; skew 500 ppm
I20260811 03:11:39.253651 16648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:11:39.256078 16648 webserver.cc:460] Webserver started at http://127.11.60.62:41057/ using document root <none> and password file <none>
I20260811 03:11:39.256799 16648 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:39.256937 16648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:39.257269 16648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:39.260264 16648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "685aee7832164f1ca867cc578627c6d7"
format_stamp: "Formatted at 2026-08-11 03:11:39 on dist-test-slave-tm1g"
server_key: "e588b51987ecc8352e9f3f6e540978e9"
server_key_iv: "649f5f0201d7500fa41ecdb13f9cb880"
server_key_version: "encryptionkey@0"
I20260811 03:11:39.261104 16648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "685aee7832164f1ca867cc578627c6d7"
format_stamp: "Formatted at 2026-08-11 03:11:39 on dist-test-slave-tm1g"
server_key: "e588b51987ecc8352e9f3f6e540978e9"
server_key_iv: "649f5f0201d7500fa41ecdb13f9cb880"
server_key_version: "encryptionkey@0"
I20260811 03:11:39.266494 16648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 03:11:39.269903 16664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:39.271286 16648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:39.271471 16648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "685aee7832164f1ca867cc578627c6d7"
format_stamp: "Formatted at 2026-08-11 03:11:39 on dist-test-slave-tm1g"
server_key: "e588b51987ecc8352e9f3f6e540978e9"
server_key_iv: "649f5f0201d7500fa41ecdb13f9cb880"
server_key_version: "encryptionkey@0"
I20260811 03:11:39.271699 16648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:39.279055 16648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:39.279855 16648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:39.280176 16648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:39.299387 16648 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:36431
I20260811 03:11:39.299446 16718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:36431 every 8 connection(s)
I20260811 03:11:39.300926 16648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:39.303444 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 16648
I20260811 03:11:39.303645 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:39.304334 11504 external_mini_cluster.cc:1428] Setting key cfa29f33adc6e21f04b515447e2352c3
I20260811 03:11:39.305109 16719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:39.314227 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:39.307023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52556 (local address 127.11.60.62:36431)
0811 03:11:39.308129 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:39.308143 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:39.308200 (+ 57us) server_negotiation.cc:407] Connection header received
0811 03:11:39.310109 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:39.310137 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:39.310719 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:39.311165 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:39.312208 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:39.312221 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:39.312237 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:39.312254 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:39.312326 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:39.312351 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:39.312607 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:39.312746 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 03:11:39.313024 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":363,"threads_started":1}
I20260811 03:11:39.319458 16719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7: Bootstrap starting.
I20260811 03:11:39.322731 16719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:39.323680 16719 log.cc:824] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:39.326293 16719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7: No bootstrap required, opened a new log
I20260811 03:11:39.333662 16719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685aee7832164f1ca867cc578627c6d7" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36431 } }
I20260811 03:11:39.334025 16719 raft_consensus.cc:374] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:39.334100 16719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 685aee7832164f1ca867cc578627c6d7, State: Initialized, Role: FOLLOWER
I20260811 03:11:39.334678 16719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685aee7832164f1ca867cc578627c6d7" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36431 } }
I20260811 03:11:39.334951 16719 raft_consensus.cc:388] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:39.335057 16719 raft_consensus.cc:482] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:39.335181 16719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:39.337744 16719 raft_consensus.cc:504] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685aee7832164f1ca867cc578627c6d7" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36431 } }
I20260811 03:11:39.338207 16719 leader_election.cc:302] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 685aee7832164f1ca867cc578627c6d7; no voters:
I20260811 03:11:39.338892 16719 leader_election.cc:288] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:39.339051 16726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:39.339818 16726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [term 1 LEADER]: Becoming Leader. State: Replica: 685aee7832164f1ca867cc578627c6d7, State: Running, Role: LEADER
I20260811 03:11:39.340369 16726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685aee7832164f1ca867cc578627c6d7" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36431 } }
I20260811 03:11:39.341030 16719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:39.343514 16728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 685aee7832164f1ca867cc578627c6d7. Latest consensus state: current_term: 1 leader_uuid: "685aee7832164f1ca867cc578627c6d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685aee7832164f1ca867cc578627c6d7" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36431 } } }
I20260811 03:11:39.343655 16727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "685aee7832164f1ca867cc578627c6d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685aee7832164f1ca867cc578627c6d7" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36431 } } }
I20260811 03:11:39.343972 16728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:39.344098 16727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:39.346786 16732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:39.352348 16732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:39.361452 16732 catalog_manager.cc:1347] Generated new cluster ID: 94c3bd7a3e0f430fb0b80d79070055be
I20260811 03:11:39.361588 16732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:39.373605 16732 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:39.374019 16732 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:39.385864 16732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7: Generated new TSK 0
I20260811 03:11:39.386976 16732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:11:39.399930 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36431
--builtin_ntp_servers=127.11.60.20:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 03:11:39.742226 16746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:39.742664 16746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:39.742851 16746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:39.752439 16746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:39.752566 16746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:39.752734 16746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:39.762398 16746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:39.763768 16746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:39.765964 16746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:39.780455 16752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:39.780545 16753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:39.780495 16755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:39.781075 16746 server_base.cc:1034] running on GCE node
I20260811 03:11:39.781694 16746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:39.782778 16746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:39.784022 16746 hybrid_clock.cc:648] HybridClock initialized: now 1786417899783984 us; error 51 us; skew 500 ppm
I20260811 03:11:39.784389 16746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:11:39.786839 16746 webserver.cc:460] Webserver started at http://127.11.60.1:44847/ using document root <none> and password file <none>
I20260811 03:11:39.787606 16746 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:39.787748 16746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:39.788110 16746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:39.791203 16746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "ac0e034ac768472087cf8db5b505d1e6"
format_stamp: "Formatted at 2026-08-11 03:11:39 on dist-test-slave-tm1g"
server_key: "7328f0a6653ef87b8c93ad7d59512447"
server_key_iv: "ab09ade3cd084c50538c6e9acac40287"
server_key_version: "encryptionkey@0"
I20260811 03:11:39.792069 16746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "ac0e034ac768472087cf8db5b505d1e6"
format_stamp: "Formatted at 2026-08-11 03:11:39 on dist-test-slave-tm1g"
server_key: "7328f0a6653ef87b8c93ad7d59512447"
server_key_iv: "ab09ade3cd084c50538c6e9acac40287"
server_key_version: "encryptionkey@0"
I20260811 03:11:39.797857 16746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:11:39.801796 16761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:39.803318 16746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:39.803521 16746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "ac0e034ac768472087cf8db5b505d1e6"
format_stamp: "Formatted at 2026-08-11 03:11:39 on dist-test-slave-tm1g"
server_key: "7328f0a6653ef87b8c93ad7d59512447"
server_key_iv: "ab09ade3cd084c50538c6e9acac40287"
server_key_version: "encryptionkey@0"
I20260811 03:11:39.803759 16746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:39.809641 16746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:39.810534 16746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:39.810823 16746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:39.811802 16746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:39.813563 16746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:39.813699 16746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:39.813807 16746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:39.813872 16746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:39.847594 16746 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:46029
I20260811 03:11:39.847641 16885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:46029 every 8 connection(s)
I20260811 03:11:39.849025 16746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:39.854357 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 16746
I20260811 03:11:39.854558 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:39.855370 11504 external_mini_cluster.cc:1428] Setting key 5902da8c4f14d251a6b98757737b0e6d
I20260811 03:11:39.858644 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36431
--builtin_ntp_servers=127.11.60.20:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 03:11:39.858994 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:39.851724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:52329 (local address 127.11.60.62:36431)
0811 03:11:39.852163 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:39.852169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:39.853550 (+ 1381us) server_negotiation.cc:407] Connection header received
0811 03:11:39.856059 (+ 2509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:39.856066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:39.856143 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:39.856264 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:39.858050 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:39.858054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:39.858058 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:39.858074 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:39.858120 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:39.858132 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:39.858240 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:39.858534 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 03:11:39.858587 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":200,"threads_started":1}
I20260811 03:11:39.860142 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:39.852394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:36431 (local address 127.11.60.1:52329)
0811 03:11:39.853397 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:39.853438 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:39.855718 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:39.856683 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:39.856692 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:39.856806 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:39.857613 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:11:39.857616 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:39.857669 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:11:39.857783 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:39.858249 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:39.858260 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:39.858269 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 03:11:39.858600 (+ 331us) client_negotiation.cc:240] Negotiation successful
0811 03:11:39.858875 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":279,"threads_started":1}
I20260811 03:11:39.861639 16886 heartbeater.cc:347] Connected to a master server at 127.11.60.62:36431
I20260811 03:11:39.861953 16886 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:39.862695 16886 heartbeater.cc:511] Master 127.11.60.62:36431 requested a full tablet report, sending...
I20260811 03:11:39.864847 16682 ts_manager.cc:194] Registered new tserver with Master: ac0e034ac768472087cf8db5b505d1e6 (127.11.60.1:46029)
W20260811 03:11:40.192739 16890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:40.193039 16890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:40.193133 16890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:40.200304 16890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:40.200407 16890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:40.200516 16890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:40.209896 16890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:40.211215 16890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:40.212983 16890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:40.226696 16898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:40.226722 16895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:40.226655 16896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:40.228036 16890 server_base.cc:1034] running on GCE node
I20260811 03:11:40.228598 16890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:40.229544 16890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:40.230777 16890 hybrid_clock.cc:648] HybridClock initialized: now 1786417900230733 us; error 71 us; skew 500 ppm
I20260811 03:11:40.231119 16890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:11:40.233379 16890 webserver.cc:460] Webserver started at http://127.11.60.2:39151/ using document root <none> and password file <none>
I20260811 03:11:40.234110 16890 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:40.234248 16890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:40.234614 16890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:40.237462 16890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "fe6eb38fb0e2482ba5f63dd438152d97"
format_stamp: "Formatted at 2026-08-11 03:11:40 on dist-test-slave-tm1g"
server_key: "73bfcb6a9311e0fb0cb43b15e7ee7e90"
server_key_iv: "3724000f5698f8c524975a7430bc1b9e"
server_key_version: "encryptionkey@0"
I20260811 03:11:40.238251 16890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "fe6eb38fb0e2482ba5f63dd438152d97"
format_stamp: "Formatted at 2026-08-11 03:11:40 on dist-test-slave-tm1g"
server_key: "73bfcb6a9311e0fb0cb43b15e7ee7e90"
server_key_iv: "3724000f5698f8c524975a7430bc1b9e"
server_key_version: "encryptionkey@0"
I20260811 03:11:40.243631 16890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 03:11:40.247145 16904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:40.248512 16890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:11:40.248705 16890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "fe6eb38fb0e2482ba5f63dd438152d97"
format_stamp: "Formatted at 2026-08-11 03:11:40 on dist-test-slave-tm1g"
server_key: "73bfcb6a9311e0fb0cb43b15e7ee7e90"
server_key_iv: "3724000f5698f8c524975a7430bc1b9e"
server_key_version: "encryptionkey@0"
I20260811 03:11:40.248937 16890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:40.254565 16890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:40.255344 16890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:40.255620 16890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:40.256506 16890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:40.257905 16890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:40.258035 16890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:40.258162 16890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:40.258248 16890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:40.293205 16890 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:36035
I20260811 03:11:40.293298 17028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:36035 every 8 connection(s)
I20260811 03:11:40.294641 16890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:40.301812 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 16890
I20260811 03:11:40.301988 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:40.302721 11504 external_mini_cluster.cc:1428] Setting key 5995e140b93bcad1269e113fcdc454ba
I20260811 03:11:40.304397 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.297457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:54517 (local address 127.11.60.62:36431)
0811 03:11:40.297657 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:40.297661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:40.299206 (+ 1545us) server_negotiation.cc:407] Connection header received
0811 03:11:40.301648 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:40.301655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:40.301727 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:40.301854 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:40.303362 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:40.303368 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:40.303372 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:40.303394 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:40.303436 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:40.303452 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:40.303593 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:40.304049 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 03:11:40.304110 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 03:11:40.305673 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36431
--builtin_ntp_servers=127.11.60.20:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 03:11:40.305698 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.297851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:36431 (local address 127.11.60.2:54517)
0811 03:11:40.299031 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:40.299076 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:40.301324 (+ 2248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:40.302084 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:40.302095 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:40.302213 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:40.303019 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:11:40.303023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:40.303071 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:11:40.303173 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:40.303819 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:40.303830 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:40.303838 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 03:11:40.304103 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 03:11:40.304349 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":293,"threads_started":1}
I20260811 03:11:40.307142 17029 heartbeater.cc:347] Connected to a master server at 127.11.60.62:36431
I20260811 03:11:40.307493 17029 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:40.308197 17029 heartbeater.cc:511] Master 127.11.60.62:36431 requested a full tablet report, sending...
I20260811 03:11:40.309818 16682 ts_manager.cc:194] Registered new tserver with Master: fe6eb38fb0e2482ba5f63dd438152d97 (127.11.60.2:36035)
W20260811 03:11:40.641860 17032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:40.642231 17032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:40.642409 17032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:40.650521 17032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:40.650619 17032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:40.650727 17032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:40.659929 17032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:40.661196 17032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:40.662935 17032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:40.676329 17041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:40.676486 17037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:40.676460 17038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:40.677786 17032 server_base.cc:1034] running on GCE node
I20260811 03:11:40.678433 17032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:40.679340 17032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:40.680546 17032 hybrid_clock.cc:648] HybridClock initialized: now 1786417900680504 us; error 33 us; skew 500 ppm
I20260811 03:11:40.680817 17032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:11:40.683188 17032 webserver.cc:460] Webserver started at http://127.11.60.3:46391/ using document root <none> and password file <none>
I20260811 03:11:40.683840 17032 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:40.683925 17032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:40.684223 17032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:40.687110 17032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "98b28fdbd0474f88b55a82f26b52b4e5"
format_stamp: "Formatted at 2026-08-11 03:11:40 on dist-test-slave-tm1g"
server_key: "aacdec78dc11297448e3c65bbcda02a1"
server_key_iv: "6d1ee48de525b6d65851f0c91573a9b7"
server_key_version: "encryptionkey@0"
I20260811 03:11:40.687840 17032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "98b28fdbd0474f88b55a82f26b52b4e5"
format_stamp: "Formatted at 2026-08-11 03:11:40 on dist-test-slave-tm1g"
server_key: "aacdec78dc11297448e3c65bbcda02a1"
server_key_iv: "6d1ee48de525b6d65851f0c91573a9b7"
server_key_version: "encryptionkey@0"
I20260811 03:11:40.692982 17032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:11:40.696457 17047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:40.697822 17032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:11:40.697955 17032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "98b28fdbd0474f88b55a82f26b52b4e5"
format_stamp: "Formatted at 2026-08-11 03:11:40 on dist-test-slave-tm1g"
server_key: "aacdec78dc11297448e3c65bbcda02a1"
server_key_iv: "6d1ee48de525b6d65851f0c91573a9b7"
server_key_version: "encryptionkey@0"
I20260811 03:11:40.698148 17032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:40.703783 17032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:40.704548 17032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:40.704823 17032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:40.705735 17032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:40.707212 17032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:40.707298 17032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:40.707510 17032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:40.707612 17032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:40.743901 17032 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:41631
I20260811 03:11:40.743948 17174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:41631 every 8 connection(s)
I20260811 03:11:40.745280 17032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:40.749686 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 17032
I20260811 03:11:40.749856 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:40.750499 11504 external_mini_cluster.cc:1428] Setting key 80e7c652f63b035e62c9ec7196f0288b
I20260811 03:11:40.754750 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.748330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:43631 (local address 127.11.60.62:36431)
0811 03:11:40.748510 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:40.748514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:40.750238 (+ 1724us) server_negotiation.cc:407] Connection header received
0811 03:11:40.752298 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:40.752310 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:40.752400 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:40.752529 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:40.754041 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:40.754045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:40.754048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:40.754065 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:40.754103 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:40.754115 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:40.754207 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:40.754476 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 03:11:40.754530 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 03:11:40.755682 17177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.748850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:36431 (local address 127.11.60.3:43631)
0811 03:11:40.750065 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:40.750113 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:40.751961 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:40.752819 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:40.752833 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:40.752966 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:40.753718 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:11:40.753721 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:40.753761 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:11:40.753847 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:40.754208 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:40.754215 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:40.754221 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 03:11:40.754495 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 03:11:40.754706 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":342,"threads_started":1}
I20260811 03:11:40.756991 17175 heartbeater.cc:347] Connected to a master server at 127.11.60.62:36431
I20260811 03:11:40.757297 17175 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:40.757885 17175 heartbeater.cc:511] Master 127.11.60.62:36431 requested a full tablet report, sending...
I20260811 03:11:40.759363 16682 ts_manager.cc:194] Registered new tserver with Master: 98b28fdbd0474f88b55a82f26b52b4e5 (127.11.60.3:41631)
I20260811 03:11:40.766443 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:11:40.773497 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.771162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.11.60.62:36431)
0811 03:11:40.771335 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:40.771340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:40.771868 (+ 528us) server_negotiation.cc:407] Connection header received
0811 03:11:40.772005 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:40.772026 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:40.772081 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:40.772170 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:40.772987 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:40.772991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:40.772994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:40.773009 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:40.773041 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:40.773052 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:40.773162 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:40.773216 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 03:11:40.773262 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 03:11:40.777624 16682 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:11:40.784629 16682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:40.802497 17187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.796517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46029 (local address 127.0.0.1:49442)
0811 03:11:40.797389 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:40.797433 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:40.797615 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:40.799478 (+ 1863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:40.799489 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:40.799606 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:40.800608 (+ 1002us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:11:40.800614 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:40.800655 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:11:40.800761 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:40.801454 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:40.801470 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:40.801479 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 03:11:40.801693 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 03:11:40.801808 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":915,"threads_started":1}
I20260811 03:11:40.803853 17188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.796783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49442 (local address 127.11.60.1:46029)
0811 03:11:40.797394 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:40.797406 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:40.797785 (+ 379us) server_negotiation.cc:407] Connection header received
0811 03:11:40.797867 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:40.797878 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:40.798163 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:40.798369 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:40.801006 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:40.801019 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:40.801035 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:40.801070 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:40.801149 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:40.801176 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:40.801350 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:40.803027 (+ 1677us) server_negotiation.cc:299] Negotiation successful
0811 03:11:40.803155 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":285,"threads_started":1}
I20260811 03:11:40.808095 17187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.797308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:41631 (local address 127.0.0.1:46878)
0811 03:11:40.802739 (+ 5431us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:40.802756 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:40.802826 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:40.804937 (+ 2111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:40.804945 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:40.805008 (+ 63us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:40.805708 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:11:40.805714 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:40.805737 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:11:40.805800 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:40.807494 (+ 1694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:40.807502 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:40.807511 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 03:11:40.807628 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 03:11:40.807698 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5349,"threads_started":1}
I20260811 03:11:40.809391 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.797714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46878 (local address 127.11.60.3:41631)
0811 03:11:40.804258 (+ 6544us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:40.804271 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:40.804313 (+ 42us) server_negotiation.cc:407] Connection header received
0811 03:11:40.804404 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:40.804415 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:40.804702 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:40.805393 (+ 691us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:40.807200 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:40.807207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:40.807221 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:40.807243 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:40.807312 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:40.807337 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:40.808642 (+ 1305us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:40.808731 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 03:11:40.808854 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6330,"thread_start_us":2528,"threads_started":1}
I20260811 03:11:40.811528 17187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.798332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:36035 (local address 127.0.0.1:54754)
0811 03:11:40.808283 (+ 9951us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:40.808303 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:40.808383 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:40.809128 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:40.809136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:40.809171 (+ 35us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:40.809801 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:11:40.809808 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:40.809838 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:11:40.809886 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:40.811099 (+ 1213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:40.811104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:40.811109 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 03:11:40.811196 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 03:11:40.811247 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9855,"thread_start_us":11518,"threads_started":1}
I20260811 03:11:40.812172 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:40.798966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54754 (local address 127.11.60.2:36035)
0811 03:11:40.808394 (+ 9428us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:40.808407 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:40.808450 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:11:40.808540 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:40.808551 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:40.808812 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:40.809524 (+ 712us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:40.810809 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:40.810819 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:40.810845 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:40.810877 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:40.810941 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:40.810965 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:40.811805 (+ 840us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:40.811886 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 03:11:40.811960 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9220,"thread_start_us":9090,"threads_started":1}
I20260811 03:11:40.814456 17103 tablet_service.cc:1467] Processing CreateTablet for tablet c7571dfeeb164b41af69993ffb9d6e41 (DEFAULT_TABLE table=test-table [id=e5a8314bb61b4312bc0bcb05655f9202]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:40.814456 16815 tablet_service.cc:1467] Processing CreateTablet for tablet c7571dfeeb164b41af69993ffb9d6e41 (DEFAULT_TABLE table=test-table [id=e5a8314bb61b4312bc0bcb05655f9202]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:40.815582 16954 tablet_service.cc:1467] Processing CreateTablet for tablet c7571dfeeb164b41af69993ffb9d6e41 (DEFAULT_TABLE table=test-table [id=e5a8314bb61b4312bc0bcb05655f9202]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:40.815999 16815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7571dfeeb164b41af69993ffb9d6e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:40.815997 17103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7571dfeeb164b41af69993ffb9d6e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:40.816661 16954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7571dfeeb164b41af69993ffb9d6e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:40.828751 17193 tablet_bootstrap.cc:492] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: Bootstrap starting.
I20260811 03:11:40.830922 17195 tablet_bootstrap.cc:492] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: Bootstrap starting.
I20260811 03:11:40.832633 17194 tablet_bootstrap.cc:492] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6: Bootstrap starting.
I20260811 03:11:40.833303 17193 tablet_bootstrap.cc:654] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:40.834532 17195 tablet_bootstrap.cc:654] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:40.834908 17193 log.cc:824] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:40.835769 17195 log.cc:824] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:40.835925 17194 tablet_bootstrap.cc:654] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:40.837162 17194 log.cc:824] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:40.837985 17193 tablet_bootstrap.cc:492] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: No bootstrap required, opened a new log
I20260811 03:11:40.838274 17193 ts_tablet_manager.cc:1397] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 03:11:40.838860 17195 tablet_bootstrap.cc:492] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: No bootstrap required, opened a new log
I20260811 03:11:40.839121 17195 ts_tablet_manager.cc:1397] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260811 03:11:40.839625 17194 tablet_bootstrap.cc:492] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6: No bootstrap required, opened a new log
I20260811 03:11:40.839867 17194 ts_tablet_manager.cc:1397] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 03:11:40.847728 17194 raft_consensus.cc:348] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0e034ac768472087cf8db5b505d1e6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46029 } } peers { permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 } } peers { permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 } }
I20260811 03:11:40.848170 17194 raft_consensus.cc:374] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:40.848258 17194 raft_consensus.cc:729] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac0e034ac768472087cf8db5b505d1e6, State: Initialized, Role: FOLLOWER
I20260811 03:11:40.848896 17194 consensus_queue.cc:260] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0e034ac768472087cf8db5b505d1e6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46029 } } peers { permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 } } peers { permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 } }
I20260811 03:11:40.849665 17195 raft_consensus.cc:348] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0e034ac768472087cf8db5b505d1e6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46029 } } peers { permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 } } peers { permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 } }
I20260811 03:11:40.849648 17193 raft_consensus.cc:348] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0e034ac768472087cf8db5b505d1e6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46029 } } peers { permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 } } peers { permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 } }
I20260811 03:11:40.850075 17193 raft_consensus.cc:374] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:40.850080 17195 raft_consensus.cc:374] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:40.850168 17195 raft_consensus.cc:729] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98b28fdbd0474f88b55a82f26b52b4e5, State: Initialized, Role: FOLLOWER
I20260811 03:11:40.850168 17193 raft_consensus.cc:729] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6eb38fb0e2482ba5f63dd438152d97, State: Initialized, Role: FOLLOWER
I20260811 03:11:40.850885 17193 consensus_queue.cc:260] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0e034ac768472087cf8db5b505d1e6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46029 } } peers { permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 } } peers { permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 } }
I20260811 03:11:40.850878 17195 consensus_queue.cc:260] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0e034ac768472087cf8db5b505d1e6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46029 } } peers { permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 } } peers { permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 } }
I20260811 03:11:40.856364 17193 ts_tablet_manager.cc:1428] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
I20260811 03:11:40.856834 17029 heartbeater.cc:503] Master 127.11.60.62:36431 was elected leader, sending a full tablet report...
I20260811 03:11:40.867208 16886 heartbeater.cc:503] Master 127.11.60.62:36431 was elected leader, sending a full tablet report...
I20260811 03:11:40.867681 17175 heartbeater.cc:503] Master 127.11.60.62:36431 was elected leader, sending a full tablet report...
I20260811 03:11:40.868376 17195 ts_tablet_manager.cc:1428] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: Time spent starting tablet: real 0.029s user 0.017s sys 0.012s
I20260811 03:11:40.868570 17194 ts_tablet_manager.cc:1428] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6: Time spent starting tablet: real 0.029s user 0.020s sys 0.009s
W20260811 03:11:40.997277 17176 tablet.cc:2369] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:41.031957 17199 raft_consensus.cc:482] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:41.032342 17199 raft_consensus.cc:504] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0e034ac768472087cf8db5b505d1e6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46029 } } peers { permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 } } peers { permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 } }
I20260811 03:11:41.034173 17199 leader_election.cc:288] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe6eb38fb0e2482ba5f63dd438152d97 (127.11.60.2:36035), 98b28fdbd0474f88b55a82f26b52b4e5 (127.11.60.3:41631)
I20260811 03:11:41.037519 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.034510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:36445 (local address 127.11.60.2:36035)
0811 03:11:41.034713 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:41.034721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:41.035473 (+ 752us) server_negotiation.cc:407] Connection header received
0811 03:11:41.035682 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:41.035687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:41.035736 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:41.035843 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:41.036863 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:41.036869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:41.036874 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:41.036894 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:41.036931 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:41.036943 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:41.037064 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:41.037152 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 03:11:41.037219 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 03:11:41.037632 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.034718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:55259 (local address 127.11.60.3:41631)
0811 03:11:41.034896 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:41.034904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:41.035846 (+ 942us) server_negotiation.cc:407] Connection header received
0811 03:11:41.036029 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:41.036038 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:41.036089 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:41.036194 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:41.036981 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:41.036986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:41.036990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:41.037018 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:41.037048 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:41.037059 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:41.037191 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:41.037298 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 03:11:41.037365 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 03:11:41.037706 17202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.034482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:36035 (local address 127.11.60.1:36445)
0811 03:11:41.035359 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:41.035375 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:41.035527 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:41.035870 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:41.035874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:41.035899 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:41.036626 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:11:41.036631 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:41.036652 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:11:41.036705 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:41.037043 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:41.037048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:41.037053 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 03:11:41.037196 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 03:11:41.037246 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":501,"threads_started":1}
I20260811 03:11:41.038203 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.034569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:41631 (local address 127.11.60.1:55259)
0811 03:11:41.035722 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:41.035736 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:41.035876 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:41.036215 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:41.036220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:41.036244 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:41.036785 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:11:41.036788 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:41.036801 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:11:41.036835 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:41.037158 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:41.037162 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:41.037168 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 03:11:41.037331 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 03:11:41.037383 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1064,"thread_start_us":508,"threads_started":1}
I20260811 03:11:41.038775 16975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7571dfeeb164b41af69993ffb9d6e41" candidate_uuid: "ac0e034ac768472087cf8db5b505d1e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" is_pre_election: true
I20260811 03:11:41.039206 17126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7571dfeeb164b41af69993ffb9d6e41" candidate_uuid: "ac0e034ac768472087cf8db5b505d1e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" is_pre_election: true
I20260811 03:11:41.039294 16975 raft_consensus.cc:2454] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac0e034ac768472087cf8db5b505d1e6 in term 0.
I20260811 03:11:41.039664 17126 raft_consensus.cc:2454] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac0e034ac768472087cf8db5b505d1e6 in term 0.
I20260811 03:11:41.039990 16762 leader_election.cc:302] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac0e034ac768472087cf8db5b505d1e6, fe6eb38fb0e2482ba5f63dd438152d97; no voters:
I20260811 03:11:41.040403 17199 raft_consensus.cc:2781] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:41.040566 17199 raft_consensus.cc:482] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:41.040678 17199 raft_consensus.cc:3037] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:41.043076 17199 raft_consensus.cc:504] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0e034ac768472087cf8db5b505d1e6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46029 } } peers { permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 } } peers { permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 } }
I20260811 03:11:41.043717 17199 leader_election.cc:288] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [CANDIDATE]: Term 1 election: Requested vote from peers fe6eb38fb0e2482ba5f63dd438152d97 (127.11.60.2:36035), 98b28fdbd0474f88b55a82f26b52b4e5 (127.11.60.3:41631)
I20260811 03:11:41.044090 16975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7571dfeeb164b41af69993ffb9d6e41" candidate_uuid: "ac0e034ac768472087cf8db5b505d1e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6eb38fb0e2482ba5f63dd438152d97"
I20260811 03:11:41.044121 17126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7571dfeeb164b41af69993ffb9d6e41" candidate_uuid: "ac0e034ac768472087cf8db5b505d1e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98b28fdbd0474f88b55a82f26b52b4e5"
I20260811 03:11:41.044287 16975 raft_consensus.cc:3037] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:41.044298 17126 raft_consensus.cc:3037] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:41.046591 17126 raft_consensus.cc:2454] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac0e034ac768472087cf8db5b505d1e6 in term 1.
I20260811 03:11:41.046624 16975 raft_consensus.cc:2454] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac0e034ac768472087cf8db5b505d1e6 in term 1.
W20260811 03:11:41.046835 17030 tablet.cc:2369] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:41.047091 16762 leader_election.cc:302] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac0e034ac768472087cf8db5b505d1e6, fe6eb38fb0e2482ba5f63dd438152d97; no voters:
I20260811 03:11:41.047429 17199 raft_consensus.cc:2781] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:41.047976 17199 raft_consensus.cc:686] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 1 LEADER]: Becoming Leader. State: Replica: ac0e034ac768472087cf8db5b505d1e6, State: Running, Role: LEADER
I20260811 03:11:41.048439 17199 consensus_queue.cc:237] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0e034ac768472087cf8db5b505d1e6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46029 } } peers { permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 } } peers { permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 } }
I20260811 03:11:41.052984 16682 catalog_manager.cc:5495] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac0e034ac768472087cf8db5b505d1e6 (127.11.60.1). New cstate: current_term: 1 leader_uuid: "ac0e034ac768472087cf8db5b505d1e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0e034ac768472087cf8db5b505d1e6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 } health_report { overall_health: UNKNOWN } } }
W20260811 03:11:41.101612 16887 tablet.cc:2369] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:41.116492 17188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.114162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49454 (local address 127.11.60.1:46029)
0811 03:11:41.114390 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:41.114396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:41.114503 (+ 107us) server_negotiation.cc:407] Connection header received
0811 03:11:41.114777 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:41.114782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:41.114862 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:41.114975 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:41.115856 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:41.115864 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:41.115873 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:41.115889 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:41.115922 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:41.115934 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:41.116063 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:41.116152 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 03:11:41.116215 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 03:11:41.124467 17126 raft_consensus.cc:1261] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5 [term 1 FOLLOWER]: Refusing update from remote peer ac0e034ac768472087cf8db5b505d1e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:41.124501 16975 raft_consensus.cc:1261] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97 [term 1 FOLLOWER]: Refusing update from remote peer ac0e034ac768472087cf8db5b505d1e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:41.125414 17199 consensus_queue.cc:1035] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98b28fdbd0474f88b55a82f26b52b4e5" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:41.125705 17204 consensus_queue.cc:1035] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6eb38fb0e2482ba5f63dd438152d97" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 36035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:41.137281 17208 mvcc.cc:205] Tried to move back new op lower bound from 7317167722996387840 to 7317167722698313728. Current Snapshot: MvccSnapshot[applied={T|T < 7317167722996387840}]
I20260811 03:11:41.137589 17209 mvcc.cc:205] Tried to move back new op lower bound from 7317167722996387840 to 7317167722698313728. Current Snapshot: MvccSnapshot[applied={T|T < 7317167722996387840}]
I20260811 03:11:41.150676 16681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:41.151001 16681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:41.155109 16681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 685aee7832164f1ca867cc578627c6d7: Sending DeleteTablet for 3 replicas of tablet c7571dfeeb164b41af69993ffb9d6e41
I20260811 03:11:41.156359 17103 tablet_service.cc:1514] Processing DeleteTablet for tablet c7571dfeeb164b41af69993ffb9d6e41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:41 UTC) from {username='slave'} at 127.0.0.1:46878
I20260811 03:11:41.156706 16815 tablet_service.cc:1514] Processing DeleteTablet for tablet c7571dfeeb164b41af69993ffb9d6e41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:41 UTC) from {username='slave'} at 127.0.0.1:49442
I20260811 03:11:41.156759 16954 tablet_service.cc:1514] Processing DeleteTablet for tablet c7571dfeeb164b41af69993ffb9d6e41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:41 UTC) from {username='slave'} at 127.0.0.1:54754
I20260811 03:11:41.158666 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 16746
I20260811 03:11:41.172926 17219 tablet_replica.cc:330] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: stopping tablet replica
I20260811 03:11:41.173688 17219 raft_consensus.cc:2229] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:41.174461 17219 raft_consensus.cc:2258] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:41.175446 17188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.159723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49466 (local address 127.11.60.1:46029)
0811 03:11:41.160181 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:41.160190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:41.165364 (+ 5174us) server_negotiation.cc:407] Connection header received
0811 03:11:41.165624 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:41.165633 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:41.165718 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:41.165843 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:41.174293 (+ 8450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:41.174318 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:41.174322 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:41.174347 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:41.174390 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:41.174414 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:41.174528 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:41.175062 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 03:11:41.175138 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
I20260811 03:11:41.176681 16881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:41.177661 17221 tablet_replica.cc:330] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: stopping tablet replica
I20260811 03:11:41.178107 17221 raft_consensus.cc:2229] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:41.178287 17219 ts_tablet_manager.cc:1905] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:41.178678 17221 raft_consensus.cc:2258] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:41.181023 17221 ts_tablet_manager.cc:1905] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:41.186156 17219 ts_tablet_manager.cc:1918] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:41.186352 17219 log.cc:1196] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/c7571dfeeb164b41af69993ffb9d6e41
I20260811 03:11:41.186919 17219 ts_tablet_manager.cc:1939] T c7571dfeeb164b41af69993ffb9d6e41 P 98b28fdbd0474f88b55a82f26b52b4e5: Deleting consensus metadata
I20260811 03:11:41.188298 17221 ts_tablet_manager.cc:1918] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:41.188469 17221 log.cc:1196] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/c7571dfeeb164b41af69993ffb9d6e41
I20260811 03:11:41.188956 16668 catalog_manager.cc:4841] TS 98b28fdbd0474f88b55a82f26b52b4e5 (127.11.60.3:41631): tablet c7571dfeeb164b41af69993ffb9d6e41 (table test-table [id=e5a8314bb61b4312bc0bcb05655f9202]) successfully deleted
I20260811 03:11:41.189096 17221 ts_tablet_manager.cc:1939] T c7571dfeeb164b41af69993ffb9d6e41 P fe6eb38fb0e2482ba5f63dd438152d97: Deleting consensus metadata
I20260811 03:11:41.190917 16665 catalog_manager.cc:4841] TS fe6eb38fb0e2482ba5f63dd438152d97 (127.11.60.2:36035): tablet c7571dfeeb164b41af69993ffb9d6e41 (table test-table [id=e5a8314bb61b4312bc0bcb05655f9202]) successfully deleted
I20260811 03:11:41.284478 17220 tablet_replica.cc:330] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6: stopping tablet replica
I20260811 03:11:41.285068 17220 raft_consensus.cc:2229] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:11:41.285811 17220 raft_consensus.cc:2258] T c7571dfeeb164b41af69993ffb9d6e41 P ac0e034ac768472087cf8db5b505d1e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:41.286998 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 16746
W20260811 03:11:41.313862 16665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.60.1:46029 (error 108)
I20260811 03:11:41.314435 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 16890
I20260811 03:11:41.316356 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.315571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46029 (local address 127.0.0.1:49480)
0811 03:11:41.315996 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:41.316145 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":220,"threads_started":1}
W20260811 03:11:41.316643 16665 catalog_manager.cc:4584] TS ac0e034ac768472087cf8db5b505d1e6 (127.11.60.1:46029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c7571dfeeb164b41af69993ffb9d6e41: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46029: connect: Connection refused (error 111)
I20260811 03:11:41.317960 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.315253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54762 (local address 127.11.60.2:36035)
0811 03:11:41.315440 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:41.315447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:41.315468 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:11:41.315608 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:41.315614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:41.315739 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:41.315865 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:41.317310 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:41.317314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:41.317317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:41.317332 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:41.317364 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:41.317377 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:41.317478 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:41.317605 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 03:11:41.317653 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 03:11:41.318848 17023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:41.370376 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.369823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46029 (local address 127.0.0.1:49484)
0811 03:11:41.370057 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:41.370169 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 03:11:41.371398 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.371042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46029 (local address 127.0.0.1:49500)
0811 03:11:41.371192 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:41.371268 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 03:11:41.410416 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 16890
I20260811 03:11:41.430965 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 17032
I20260811 03:11:41.432785 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.432277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46029 (local address 127.0.0.1:49506)
0811 03:11:41.432492 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:41.432594 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 03:11:41.434010 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.433546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46029 (local address 127.0.0.1:49514)
0811 03:11:41.433740 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:41.433828 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 03:11:41.434099 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.431785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46892 (local address 127.11.60.3:41631)
0811 03:11:41.431981 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:41.431985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:41.432009 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:11:41.432209 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:41.432213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:41.432308 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:41.432445 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:41.433407 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:41.433419 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:41.433423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:41.433437 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:41.433469 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:41.433481 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:41.433611 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:41.433736 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 03:11:41.433800 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 03:11:41.434758 17170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:41.514757 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.514219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46029 (local address 127.0.0.1:49516)
0811 03:11:41.514500 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:41.514611 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 03:11:41.515872 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:41.515446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46029 (local address 127.0.0.1:49526)
0811 03:11:41.515659 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:41.515727 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 03:11:41.523190 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 17032
I20260811 03:11:41.542107 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 16648
I20260811 03:11:41.543341 16714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:41.652179 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 16648
2026-08-11T03:11:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2845 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 03:11:41.697101 11504 mini_oidc.cc:152] Starting JWKS server
I20260811 03:11:41.700027 11504 webserver.cc:460] Webserver started at https://127.0.0.1:42307/ using document root <none> and password file <none>
I20260811 03:11:41.700347 11504 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 03:11:41.700977 11504 webserver.cc:460] Webserver started at http://127.0.0.1:32895/ using document root <none> and password file <none>
2026-08-11T03:11:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:41Z Disabled control of system clock
I20260811 03:11:41.734584 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:36235
--webserver_interface=127.11.60.62
--webserver_port=0
--builtin_ntp_servers=127.11.60.20:46227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:36235
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42307/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/testchainca.cert with env {}
W20260811 03:11:42.087376 17237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:42.087680 17237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:42.087826 17237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:42.095065 17237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:42.095170 17237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:42.095239 17237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:42.095333 17237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:42.095419 17237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 03:11:42.095532 17237 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 03:11:42.095587 17237 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42307/jwks.json
I20260811 03:11:42.106006 17237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:36235
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:36235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42307/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:42.107460 17237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:42.108862 17237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:42.120905 17242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:42.120857 17245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:42.120937 17243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:42.122064 17237 server_base.cc:1034] running on GCE node
I20260811 03:11:42.123031 17237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:42.124454 17237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:42.125654 17237 hybrid_clock.cc:648] HybridClock initialized: now 1786417902125618 us; error 37 us; skew 500 ppm
I20260811 03:11:42.126025 17237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:11:42.128293 17237 webserver.cc:460] Webserver started at http://127.11.60.62:42477/ using document root <none> and password file <none>
I20260811 03:11:42.129060 17237 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:42.129185 17237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:42.129524 17237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:42.132509 17237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "c637d3ca32a9445f8b83300547b3ff71"
format_stamp: "Formatted at 2026-08-11 03:11:42 on dist-test-slave-tm1g"
server_key: "4540837c8615dc6f25342e01f042236e"
server_key_iv: "74f80a816d373715f1abe1e1e1384153"
server_key_version: "encryptionkey@0"
I20260811 03:11:42.133360 17237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "c637d3ca32a9445f8b83300547b3ff71"
format_stamp: "Formatted at 2026-08-11 03:11:42 on dist-test-slave-tm1g"
server_key: "4540837c8615dc6f25342e01f042236e"
server_key_iv: "74f80a816d373715f1abe1e1e1384153"
server_key_version: "encryptionkey@0"
I20260811 03:11:42.138772 17237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 03:11:42.142406 17251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:42.143736 17237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:11:42.143947 17237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "c637d3ca32a9445f8b83300547b3ff71"
format_stamp: "Formatted at 2026-08-11 03:11:42 on dist-test-slave-tm1g"
server_key: "4540837c8615dc6f25342e01f042236e"
server_key_iv: "74f80a816d373715f1abe1e1e1384153"
server_key_version: "encryptionkey@0"
I20260811 03:11:42.144176 17237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:42.158838 17237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:42.159646 17237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:42.159967 17237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:42.178504 17237 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:36235
I20260811 03:11:42.178524 17308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:36235 every 8 connection(s)
I20260811 03:11:42.179935 17237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:42.183735 17309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:42.186493 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 17237
I20260811 03:11:42.186729 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:42.187559 11504 external_mini_cluster.cc:1428] Setting key 6f6aa956ac3ff6450f1e042bda680944
I20260811 03:11:42.195825 17309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71: Bootstrap starting.
I20260811 03:11:42.199015 17309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:42.200099 17309 log.cc:824] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:42.201623 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:42.189632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.11.60.62:36235)
0811 03:11:42.190760 (+ 1128us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:42.190772 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:42.190813 (+ 41us) server_negotiation.cc:407] Connection header received
0811 03:11:42.192483 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:42.192515 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:42.192973 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:42.193358 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:42.193974 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:42.195331 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:42.196733 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:42.197432 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:42.197650 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:42.198900 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:42.198908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:42.198925 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:42.199547 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:42.199639 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:42.199661 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:42.199797 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:42.200019 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 03:11:42.200351 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":418,"threads_started":1}
I20260811 03:11:42.203094 17309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71: No bootstrap required, opened a new log
I20260811 03:11:42.205420 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:11:42.211024 17309 raft_consensus.cc:348] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c637d3ca32a9445f8b83300547b3ff71" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36235 } }
I20260811 03:11:42.211397 17309 raft_consensus.cc:374] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:42.211524 17309 raft_consensus.cc:729] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c637d3ca32a9445f8b83300547b3ff71, State: Initialized, Role: FOLLOWER
I20260811 03:11:42.212123 17309 consensus_queue.cc:260] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c637d3ca32a9445f8b83300547b3ff71" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36235 } }
I20260811 03:11:42.212352 17309 raft_consensus.cc:388] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:42.212512 17309 raft_consensus.cc:482] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:42.212690 17309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:42.215392 17309 raft_consensus.cc:504] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c637d3ca32a9445f8b83300547b3ff71" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36235 } }
I20260811 03:11:42.215909 17309 leader_election.cc:302] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c637d3ca32a9445f8b83300547b3ff71; no voters:
I20260811 03:11:42.216496 17309 leader_election.cc:288] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:42.216710 17314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:42.217437 17314 raft_consensus.cc:686] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [term 1 LEADER]: Becoming Leader. State: Replica: c637d3ca32a9445f8b83300547b3ff71, State: Running, Role: LEADER
I20260811 03:11:42.217980 17314 consensus_queue.cc:237] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c637d3ca32a9445f8b83300547b3ff71" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36235 } }
I20260811 03:11:42.218688 17309 sys_catalog.cc:564] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:42.221081 17316 sys_catalog.cc:455] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c637d3ca32a9445f8b83300547b3ff71. Latest consensus state: current_term: 1 leader_uuid: "c637d3ca32a9445f8b83300547b3ff71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c637d3ca32a9445f8b83300547b3ff71" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36235 } } }
I20260811 03:11:42.221977 17316 sys_catalog.cc:458] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:42.221212 17315 sys_catalog.cc:455] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c637d3ca32a9445f8b83300547b3ff71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c637d3ca32a9445f8b83300547b3ff71" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36235 } } }
I20260811 03:11:42.222700 17315 sys_catalog.cc:458] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:42.226279 17323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:42.230651 17323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:42.238173 17323 catalog_manager.cc:1347] Generated new cluster ID: 612f0c06aa364d3aba9c405c53efc8b7
I20260811 03:11:42.238274 17323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:42.275115 17323 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:42.276427 17323 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:42.283479 17323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c637d3ca32a9445f8b83300547b3ff71: Generated new TSK 0
I20260811 03:11:42.284641 17323 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 03:11:42.327684 17340 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 03:11:42.327911 17340 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 03:11:42.329236 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:42.305961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57356 (local address 127.11.60.62:36235)
0811 03:11:42.306182 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:42.306189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:42.306718 (+ 529us) server_negotiation.cc:407] Connection header received
0811 03:11:42.307043 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:42.307049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:42.307146 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:42.307296 (+ 150us) server_negotiation.cc:226] Negotiated authn=JWT
0811 03:11:42.307815 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:42.308968 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:42.310432 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:42.311161 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:42.311286 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:42.311820 (+ 534us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 03:11:42.328532 (+ 16712us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 03:11:42.328924 (+ 392us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57356: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 03:11:42.329368 17312 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57356: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 03:11:42.330844 17339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:11:42.305851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:36235 (local address 127.0.0.1:57356)
0811 03:11:42.306606 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:42.306630 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:42.306777 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:42.307342 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:42.307346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:42.307356 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 03:11:42.307678 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:42.307686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:42.309110 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:42.309114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:42.310236 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:42.310247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:42.311326 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:42.311330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:42.311463 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:42.311525 (+ 62us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 03:11:42.328852 (+ 17327us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 03:11:42.329139 (+ 287us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.60.62:36235: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":389,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.60.62:36235: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 03:11:42.332878 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 17237
I20260811 03:11:42.334038 17302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:42.431083 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 17237
2026-08-11T03:11:42Z chronyd exiting
I20260811 03:11:42.484259 11504 test_util.cc:175] -----------------------------------------------
I20260811 03:11:42.484453 11504 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417858274425-11504-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (792 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 03:11:42.488493 11504 mini_oidc.cc:152] Starting JWKS server
I20260811 03:11:42.490573 11504 webserver.cc:460] Webserver started at https://127.0.0.1:40343/ using document root <none> and password file <none>
I20260811 03:11:42.490824 11504 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 03:11:42.491531 11504 webserver.cc:460] Webserver started at http://127.0.0.1:35941/ using document root <none> and password file <none>
2026-08-11T03:11:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:42Z Disabled control of system clock
I20260811 03:11:42.525455 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:39917
--webserver_interface=127.11.60.62
--webserver_port=0
--builtin_ntp_servers=127.11.60.20:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:39917
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40343/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/testchainca.cert with env {}
W20260811 03:11:42.862677 17347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:42.863042 17347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:42.863212 17347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:42.870496 17347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:42.870653 17347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:42.870757 17347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:42.870868 17347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:42.870959 17347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 03:11:42.871052 17347 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 03:11:42.871131 17347 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40343/jwks.json
I20260811 03:11:42.881151 17347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:39917
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:39917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40343/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:42.882581 17347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:42.883947 17347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:42.896092 17352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:42.896039 17353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:42.896107 17356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:42.897012 17347 server_base.cc:1034] running on GCE node
I20260811 03:11:42.897792 17347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:42.899297 17347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:42.900547 17347 hybrid_clock.cc:648] HybridClock initialized: now 1786417902900507 us; error 54 us; skew 500 ppm
I20260811 03:11:42.900916 17347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:11:42.907757 17347 webserver.cc:460] Webserver started at http://127.11.60.62:36409/ using document root <none> and password file <none>
I20260811 03:11:42.908572 17347 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:42.908725 17347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:42.909081 17347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:42.912205 17347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "415f6ab489b145a58ca3645473fd0b51"
format_stamp: "Formatted at 2026-08-11 03:11:42 on dist-test-slave-tm1g"
server_key: "a63cc2996c7a2e3b954e5a27dbea81a5"
server_key_iv: "808cf8c981904fd73d386811d8c6fb26"
server_key_version: "encryptionkey@0"
I20260811 03:11:42.913039 17347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "415f6ab489b145a58ca3645473fd0b51"
format_stamp: "Formatted at 2026-08-11 03:11:42 on dist-test-slave-tm1g"
server_key: "a63cc2996c7a2e3b954e5a27dbea81a5"
server_key_iv: "808cf8c981904fd73d386811d8c6fb26"
server_key_version: "encryptionkey@0"
I20260811 03:11:42.918267 17347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 03:11:42.921710 17363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:42.923039 17347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:42.923231 17347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "415f6ab489b145a58ca3645473fd0b51"
format_stamp: "Formatted at 2026-08-11 03:11:42 on dist-test-slave-tm1g"
server_key: "a63cc2996c7a2e3b954e5a27dbea81a5"
server_key_iv: "808cf8c981904fd73d386811d8c6fb26"
server_key_version: "encryptionkey@0"
I20260811 03:11:42.923458 17347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:42.953989 17347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:42.954911 17347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:42.955251 17347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:42.974095 17347 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:39917
I20260811 03:11:42.974115 17420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:39917 every 8 connection(s)
I20260811 03:11:42.975559 17347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:42.977324 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 17347
I20260811 03:11:42.977528 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:42.978454 11504 external_mini_cluster.cc:1428] Setting key 8c16e8b346500411bf64700df1c0ab8f
I20260811 03:11:42.980322 17421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:42.992589 17425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:42.980569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.11.60.62:39917)
0811 03:11:42.981586 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:42.981596 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:42.981638 (+ 42us) server_negotiation.cc:407] Connection header received
0811 03:11:42.983269 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:42.983291 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:42.983746 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:42.984082 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:42.984631 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:42.986012 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:42.987846 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:42.988507 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:42.988640 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:42.989953 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:42.989960 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:42.989977 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:42.990662 (+ 685us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:42.990745 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:42.990766 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:42.990876 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:42.991199 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 03:11:42.991581 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":416,"threads_started":1}
I20260811 03:11:42.993788 17421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51: Bootstrap starting.
I20260811 03:11:42.996650 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:11:42.997262 17421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:42.998252 17421 log.cc:824] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:43.001269 17421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51: No bootstrap required, opened a new log
I20260811 03:11:43.008719 17421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415f6ab489b145a58ca3645473fd0b51" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39917 } }
I20260811 03:11:43.009097 17421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:43.009212 17421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 415f6ab489b145a58ca3645473fd0b51, State: Initialized, Role: FOLLOWER
I20260811 03:11:43.009774 17421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415f6ab489b145a58ca3645473fd0b51" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39917 } }
I20260811 03:11:43.010004 17421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:43.010155 17421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:43.010355 17421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:43.012775 17421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415f6ab489b145a58ca3645473fd0b51" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39917 } }
I20260811 03:11:43.013278 17421 leader_election.cc:302] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 415f6ab489b145a58ca3645473fd0b51; no voters:
I20260811 03:11:43.013870 17421 leader_election.cc:288] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:43.014065 17427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:43.014843 17427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [term 1 LEADER]: Becoming Leader. State: Replica: 415f6ab489b145a58ca3645473fd0b51, State: Running, Role: LEADER
I20260811 03:11:43.015416 17427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415f6ab489b145a58ca3645473fd0b51" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39917 } }
I20260811 03:11:43.015970 17421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:43.018165 17428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "415f6ab489b145a58ca3645473fd0b51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415f6ab489b145a58ca3645473fd0b51" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39917 } } }
I20260811 03:11:43.018561 17428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:43.019124 17429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 415f6ab489b145a58ca3645473fd0b51. Latest consensus state: current_term: 1 leader_uuid: "415f6ab489b145a58ca3645473fd0b51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415f6ab489b145a58ca3645473fd0b51" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39917 } } }
I20260811 03:11:43.019481 17429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:43.021119 17434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:43.026504 17434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:43.036302 17434 catalog_manager.cc:1347] Generated new cluster ID: b274428ffc2f46c09bf02e51c2e534a5
I20260811 03:11:43.036463 17434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:43.095458 17434 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:43.096817 17434 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:43.104669 17434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 415f6ab489b145a58ca3645473fd0b51: Generated new TSK 0
I20260811 03:11:43.105595 17434 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 03:11:43.146155 17455 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 03:11:43.146337 17455 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 03:11:43.147359 17425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:43.119015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.11.60.62:39917)
0811 03:11:43.119256 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:43.119267 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:43.119827 (+ 560us) server_negotiation.cc:407] Connection header received
0811 03:11:43.120093 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:43.120098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:43.120215 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:43.120349 (+ 134us) server_negotiation.cc:226] Negotiated authn=JWT
0811 03:11:43.120960 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:43.121828 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:43.123982 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:43.124470 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:43.124576 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:43.125165 (+ 589us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 03:11:43.146755 (+ 21590us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 03:11:43.147107 (+ 352us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54708: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 03:11:43.147521 17425 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54708: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 03:11:43.147799 17454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:11:43.118978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39917 (local address 127.0.0.1:54708)
0811 03:11:43.119707 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:43.119729 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:43.119899 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:43.120403 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:43.120409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:43.120422 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 03:11:43.120755 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:43.120767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:43.121981 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:43.121987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:43.123835 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:43.123851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:43.124662 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:43.124671 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:43.124854 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:43.124902 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 03:11:43.147036 (+ 22134us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 03:11:43.147372 (+ 336us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.60.62:39917: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":397,"threads_started":1}
I20260811 03:11:43.149132 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 17347
I20260811 03:11:43.150449 17415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:43.249397 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 17347
2026-08-11T03:11:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (820 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 03:11:43.309119 11504 mini_oidc.cc:152] Starting JWKS server
I20260811 03:11:43.311093 11504 webserver.cc:460] Webserver started at https://127.0.0.1:34933/ using document root <none> and password file <none>
I20260811 03:11:43.311316 11504 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 03:11:43.311932 11504 webserver.cc:460] Webserver started at http://127.0.0.1:33991/ using document root <none> and password file <none>
2026-08-11T03:11:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:43Z Disabled control of system clock
I20260811 03:11:43.345734 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:37177
--webserver_interface=127.11.60.62
--webserver_port=0
--builtin_ntp_servers=127.11.60.20:44895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:37177
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34933/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 03:11:43.681988 17462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:43.682364 17462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:43.682559 17462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:43.689968 17462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:43.690078 17462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:43.690150 17462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:43.690239 17462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:43.690336 17462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 03:11:43.690435 17462 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 03:11:43.690490 17462 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34933/jwks.json
I20260811 03:11:43.700146 17462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:44895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:37177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:37177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34933/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:43.701542 17462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:43.703492 17462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:43.716385 17467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:43.716356 17470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:43.716356 17468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:43.716809 17462 server_base.cc:1034] running on GCE node
I20260811 03:11:43.717908 17462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:43.719411 17462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:43.720629 17462 hybrid_clock.cc:648] HybridClock initialized: now 1786417903720570 us; error 55 us; skew 500 ppm
I20260811 03:11:43.720949 17462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:11:43.723426 17462 webserver.cc:460] Webserver started at http://127.11.60.62:45885/ using document root <none> and password file <none>
I20260811 03:11:43.724124 17462 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:43.724212 17462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:43.724488 17462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:43.727492 17462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "8062d0b4452c4d82bc92827aed2e40ec"
format_stamp: "Formatted at 2026-08-11 03:11:43 on dist-test-slave-tm1g"
server_key: "085560df1b0ac5c1171a176ba1acb938"
server_key_iv: "39c5549007a93e4832931230cb809326"
server_key_version: "encryptionkey@0"
I20260811 03:11:43.728245 17462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "8062d0b4452c4d82bc92827aed2e40ec"
format_stamp: "Formatted at 2026-08-11 03:11:43 on dist-test-slave-tm1g"
server_key: "085560df1b0ac5c1171a176ba1acb938"
server_key_iv: "39c5549007a93e4832931230cb809326"
server_key_version: "encryptionkey@0"
I20260811 03:11:43.733466 17462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:11:43.737170 17476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:43.738571 17462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:43.738713 17462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "8062d0b4452c4d82bc92827aed2e40ec"
format_stamp: "Formatted at 2026-08-11 03:11:43 on dist-test-slave-tm1g"
server_key: "085560df1b0ac5c1171a176ba1acb938"
server_key_iv: "39c5549007a93e4832931230cb809326"
server_key_version: "encryptionkey@0"
I20260811 03:11:43.738888 17462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:43.766973 17462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:43.767797 17462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:43.768077 17462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:43.787744 17462 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:37177
I20260811 03:11:43.787765 17531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:37177 every 8 connection(s)
I20260811 03:11:43.789135 17462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:43.793085 17532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:43.799225 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 17462
I20260811 03:11:43.799472 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:43.800336 11504 external_mini_cluster.cc:1428] Setting key 227f4af53120efeb3d303d418b869312
I20260811 03:11:43.804709 17532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec: Bootstrap starting.
I20260811 03:11:43.807972 17532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:43.809006 17532 log.cc:824] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:43.812757 17532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec: No bootstrap required, opened a new log
I20260811 03:11:43.814769 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:43.802569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42644 (local address 127.11.60.62:37177)
0811 03:11:43.803525 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:43.803544 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:43.803586 (+ 42us) server_negotiation.cc:407] Connection header received
0811 03:11:43.805375 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:43.805396 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:43.805840 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:43.806241 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:43.806847 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:43.808200 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:43.809831 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:43.810837 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:43.810984 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:43.812268 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:43.812276 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:43.812290 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:43.812852 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:43.812980 (+ 128us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:43.813004 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:43.813103 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:43.813376 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 03:11:43.813716 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":308,"threads_started":1}
I20260811 03:11:43.818840 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:11:43.821681 17532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062d0b4452c4d82bc92827aed2e40ec" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37177 } }
I20260811 03:11:43.822155 17532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:43.822264 17532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8062d0b4452c4d82bc92827aed2e40ec, State: Initialized, Role: FOLLOWER
I20260811 03:11:43.822885 17532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062d0b4452c4d82bc92827aed2e40ec" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37177 } }
I20260811 03:11:43.823114 17532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:43.823261 17532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:43.823446 17532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:43.825610 17532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062d0b4452c4d82bc92827aed2e40ec" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37177 } }
I20260811 03:11:43.826215 17532 leader_election.cc:302] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8062d0b4452c4d82bc92827aed2e40ec; no voters:
I20260811 03:11:43.826874 17532 leader_election.cc:288] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:43.827030 17537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:43.827831 17537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [term 1 LEADER]: Becoming Leader. State: Replica: 8062d0b4452c4d82bc92827aed2e40ec, State: Running, Role: LEADER
I20260811 03:11:43.828413 17537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062d0b4452c4d82bc92827aed2e40ec" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37177 } }
I20260811 03:11:43.829093 17532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:43.831075 17538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8062d0b4452c4d82bc92827aed2e40ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062d0b4452c4d82bc92827aed2e40ec" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37177 } } }
I20260811 03:11:43.831395 17538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:43.832185 17539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8062d0b4452c4d82bc92827aed2e40ec. Latest consensus state: current_term: 1 leader_uuid: "8062d0b4452c4d82bc92827aed2e40ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062d0b4452c4d82bc92827aed2e40ec" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37177 } } }
I20260811 03:11:43.832544 17539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:43.833635 17543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:43.839527 17543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:43.848114 17543 catalog_manager.cc:1347] Generated new cluster ID: cb0ec6abe56c45188aff034b7c45eb30
I20260811 03:11:43.848230 17543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:43.869849 17543 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:43.871455 17543 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:43.882234 17543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8062d0b4452c4d82bc92827aed2e40ec: Generated new TSK 0
I20260811 03:11:43.883301 17543 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:11:43.915601 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:43.906211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42646 (local address 127.11.60.62:37177)
0811 03:11:43.906504 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:43.906511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:43.906745 (+ 234us) server_negotiation.cc:407] Connection header received
0811 03:11:43.906983 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:43.906991 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:43.907106 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:43.907274 (+ 168us) server_negotiation.cc:226] Negotiated authn=JWT
0811 03:11:43.907757 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:43.909050 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:43.911480 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:43.912056 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:43.912164 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:43.912813 (+ 649us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 03:11:43.914866 (+ 2053us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 03:11:43.915241 (+ 375us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42646: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 03:11:43.915851 17535 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42646: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 03:11:43.916119 17563 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:11:43.906001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37177 (local address 127.0.0.1:42646)
0811 03:11:43.906645 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:43.906661 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:43.906783 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:43.907314 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:43.907319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:43.907332 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 03:11:43.907603 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:43.907612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:43.909277 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:43.909285 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:43.911300 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:43.911318 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:43.912263 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:43.912271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:43.912515 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:43.912566 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 03:11:43.915111 (+ 2545us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 03:11:43.915418 (+ 307us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.60.62:37177: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":365,"threads_started":1}
I20260811 03:11:43.917214 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 17462
I20260811 03:11:43.918463 17527 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 03:11:43.926924 17564 webserver.cc:533] Webserver: sslize failed:
E20260811 03:11:43.927050 17564 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 03:11:44.020717 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 17462
2026-08-11T03:11:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (770 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T03:11:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:44Z Disabled control of system clock
I20260811 03:11:44.114758 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:37665
--webserver_interface=127.11.60.62
--webserver_port=0
--builtin_ntp_servers=127.11.60.20:36997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:37665
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/testchainca.cert with env {}
W20260811 03:11:44.457326 17569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:44.457628 17569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:44.457782 17569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:44.464924 17569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:44.465029 17569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:44.465121 17569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:44.465202 17569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:44.465286 17569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 03:11:44.465415 17569 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 03:11:44.465497 17569 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 03:11:44.475430 17569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:36997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:37665
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:37665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:44.476892 17569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:44.478264 17569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:44.491202 17574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:44.491149 17578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:44.491201 17575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:44.492069 17569 server_base.cc:1034] running on GCE node
I20260811 03:11:44.492901 17569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:44.494453 17569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:44.495710 17569 hybrid_clock.cc:648] HybridClock initialized: now 1786417904495653 us; error 43 us; skew 500 ppm
I20260811 03:11:44.496114 17569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:11:44.498700 17569 webserver.cc:460] Webserver started at http://127.11.60.62:43135/ using document root <none> and password file <none>
I20260811 03:11:44.499455 17569 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:44.499600 17569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:44.499950 17569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:44.502986 17569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "f17b646406bc42d1b7757cffe0ffd850"
format_stamp: "Formatted at 2026-08-11 03:11:44 on dist-test-slave-tm1g"
server_key: "cf843bd3e847c1442c1a1c8544dff606"
server_key_iv: "7053090a4fe98ca2df115cf0033ed84f"
server_key_version: "encryptionkey@0"
I20260811 03:11:44.503820 17569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "f17b646406bc42d1b7757cffe0ffd850"
format_stamp: "Formatted at 2026-08-11 03:11:44 on dist-test-slave-tm1g"
server_key: "cf843bd3e847c1442c1a1c8544dff606"
server_key_iv: "7053090a4fe98ca2df115cf0033ed84f"
server_key_version: "encryptionkey@0"
I20260811 03:11:44.509224 17569 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 03:11:44.512809 17587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:44.514195 17569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 03:11:44.514436 17569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "f17b646406bc42d1b7757cffe0ffd850"
format_stamp: "Formatted at 2026-08-11 03:11:44 on dist-test-slave-tm1g"
server_key: "cf843bd3e847c1442c1a1c8544dff606"
server_key_iv: "7053090a4fe98ca2df115cf0033ed84f"
server_key_version: "encryptionkey@0"
I20260811 03:11:44.514674 17569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:44.533579 17569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:44.534500 17569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:44.534812 17569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:44.553756 17569 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:37665
I20260811 03:11:44.553769 17638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:37665 every 8 connection(s)
I20260811 03:11:44.555186 17569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:44.558926 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 17569
I20260811 03:11:44.559230 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:44.559592 17639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:44.560024 11504 external_mini_cluster.cc:1428] Setting key e5ae11f9c26deb6e063036af6ef5dc2c
I20260811 03:11:44.573804 17642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:44.561781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51560 (local address 127.11.60.62:37665)
0811 03:11:44.563008 (+ 1227us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:44.563023 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:44.563071 (+ 48us) server_negotiation.cc:407] Connection header received
0811 03:11:44.564851 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:44.564872 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:44.565340 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:44.565682 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:44.566358 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:44.567685 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:44.569424 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:44.570053 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:44.570188 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:44.571498 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:44.571506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:44.571518 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:11:44.572096 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:44.572174 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:11:44.572194 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:44.572282 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:44.572525 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 03:11:44.572887 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":375,"threads_started":1}
I20260811 03:11:44.574292 17639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850: Bootstrap starting.
I20260811 03:11:44.577656 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:11:44.577888 17639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:44.578944 17639 log.cc:824] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:44.581931 17639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850: No bootstrap required, opened a new log
I20260811 03:11:44.589541 17639 raft_consensus.cc:348] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17b646406bc42d1b7757cffe0ffd850" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37665 } }
I20260811 03:11:44.589903 17639 raft_consensus.cc:374] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:44.590016 17639 raft_consensus.cc:729] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f17b646406bc42d1b7757cffe0ffd850, State: Initialized, Role: FOLLOWER
I20260811 03:11:44.590662 17639 consensus_queue.cc:260] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17b646406bc42d1b7757cffe0ffd850" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37665 } }
I20260811 03:11:44.590899 17639 raft_consensus.cc:388] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:44.591060 17639 raft_consensus.cc:482] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:44.591228 17639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:44.593592 17639 raft_consensus.cc:504] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17b646406bc42d1b7757cffe0ffd850" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37665 } }
I20260811 03:11:44.594107 17639 leader_election.cc:302] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f17b646406bc42d1b7757cffe0ffd850; no voters:
I20260811 03:11:44.594740 17639 leader_election.cc:288] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:44.594906 17644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:44.595664 17644 raft_consensus.cc:686] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [term 1 LEADER]: Becoming Leader. State: Replica: f17b646406bc42d1b7757cffe0ffd850, State: Running, Role: LEADER
I20260811 03:11:44.596189 17644 consensus_queue.cc:237] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17b646406bc42d1b7757cffe0ffd850" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37665 } }
I20260811 03:11:44.596922 17639 sys_catalog.cc:564] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:44.599612 17646 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f17b646406bc42d1b7757cffe0ffd850. Latest consensus state: current_term: 1 leader_uuid: "f17b646406bc42d1b7757cffe0ffd850" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17b646406bc42d1b7757cffe0ffd850" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37665 } } }
I20260811 03:11:44.599581 17645 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f17b646406bc42d1b7757cffe0ffd850" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17b646406bc42d1b7757cffe0ffd850" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37665 } } }
I20260811 03:11:44.599934 17646 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:44.599965 17645 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:44.601519 17654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:44.606768 17654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:44.616255 17654 catalog_manager.cc:1347] Generated new cluster ID: 2adc176c67b7446b91af70ba7475987c
I20260811 03:11:44.616396 17654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:44.637650 17654 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:44.639846 17654 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:44.657868 17654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f17b646406bc42d1b7757cffe0ffd850: Generated new TSK 0
I20260811 03:11:44.659091 17654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:11:44.750833 17642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:44.682742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.11.60.62:37665)
0811 03:11:44.682926 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:44.682933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:44.683499 (+ 566us) server_negotiation.cc:407] Connection header received
0811 03:11:44.683754 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:44.683761 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:44.683915 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:44.684097 (+ 182us) server_negotiation.cc:226] Negotiated authn=JWT
0811 03:11:44.684896 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:44.686085 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:44.688651 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:44.689392 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:44.689529 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:44.690179 (+ 650us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 03:11:44.750086 (+ 59907us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 03:11:44.750588 (+ 502us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51572: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 03:11:44.750998 17642 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51572: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 03:11:44.751191 17671 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:11:44.682644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37665 (local address 127.0.0.1:51572)
0811 03:11:44.683369 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:44.683392 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:44.683561 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:44.684303 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:44.684310 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:44.684327 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0811 03:11:44.684663 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:44.684678 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:44.686294 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:44.686327 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:44.688466 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:44.688484 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:44.689586 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:44.689592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:44.689820 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:44.689873 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 03:11:44.750438 (+ 60565us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 03:11:44.750789 (+ 351us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.60.62:37665: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":401,"threads_started":1}
I20260811 03:11:44.752538 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 17569
I20260811 03:11:44.753847 17634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:44.856094 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 17569
2026-08-11T03:11:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (814 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (433 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (437 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:11:45 dist-test-slave-tm1g krb5kdc[17686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:11:45 dist-test-slave-tm1g krb5kdc[17686](info): set up 2 sockets
Aug 11 03:11:45 dist-test-slave-tm1g krb5kdc[17686](info): commencing operation
krb5kdc: starting...
W20260811 03:11:46.891665 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:11:47 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:11:47.154673 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:35403
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:35403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:47.497984 17699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:47.498464 17699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:47.498665 17699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:47.506184 17699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:47.506366 17699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:47.506469 17699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:47.506582 17699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:47.506654 17699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:47.516987 17699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:35403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:35403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:47.518576 17699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:47.519979 17699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:47.532003 17704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:47.532027 17708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:47.532037 17706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:47.533201 17699 server_base.cc:1034] running on GCE node
I20260811 03:11:47.534009 17699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:47.535492 17699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:47.536700 17699 hybrid_clock.cc:648] HybridClock initialized: now 1786417907536648 us; error 37 us; skew 500 ppm
I20260811 03:11:47.537050 17699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:47 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417907, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:47.540271 17699 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:11:47.541838 17699 webserver.cc:460] Webserver started at http://127.11.60.62:36667/ using document root <none> and password file <none>
I20260811 03:11:47.542646 17699 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:47.542794 17699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:47.543128 17699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:47.546226 17699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "eb68e5481330445e88c4a9f45d61d282"
format_stamp: "Formatted at 2026-08-11 03:11:47 on dist-test-slave-tm1g"
server_key: "3b2c096cf40191708ea41dd6ce61d3cb"
server_key_iv: "b8f92e7188e115a7156890ca3bc3d763"
server_key_version: "encryptionkey@0"
I20260811 03:11:47.547133 17699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "eb68e5481330445e88c4a9f45d61d282"
format_stamp: "Formatted at 2026-08-11 03:11:47 on dist-test-slave-tm1g"
server_key: "3b2c096cf40191708ea41dd6ce61d3cb"
server_key_iv: "b8f92e7188e115a7156890ca3bc3d763"
server_key_version: "encryptionkey@0"
I20260811 03:11:47.552317 17699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:11:47.555792 17715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:47.557096 17699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:11:47.557303 17699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "eb68e5481330445e88c4a9f45d61d282"
format_stamp: "Formatted at 2026-08-11 03:11:47 on dist-test-slave-tm1g"
server_key: "3b2c096cf40191708ea41dd6ce61d3cb"
server_key_iv: "b8f92e7188e115a7156890ca3bc3d763"
server_key_version: "encryptionkey@0"
I20260811 03:11:47.557528 17699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:47.581759 17699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:47.586555 17699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:47.586869 17699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:47.606185 17699 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:35403
I20260811 03:11:47.606222 17776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:35403 every 8 connection(s)
I20260811 03:11:47.607646 17699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:47.610646 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 17699
I20260811 03:11:47.610877 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:47.611580 11504 external_mini_cluster.cc:1428] Setting key 11062346de2bbb5aa48e37fce44bf9e1
I20260811 03:11:47.611878 17777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:47.627039 17777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282: Bootstrap starting.
Aug 11 03:11:47 dist-test-slave-tm1g krb5kdc[17686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:47.631201 17777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:47.632601 17777 log.cc:824] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:47.636282 17777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282: No bootstrap required, opened a new log
I20260811 03:11:47.637504 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:47.613677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.11.60.62:35403)
0811 03:11:47.614546 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:47.614557 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:47.614600 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:11:47.616713 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:47.616735 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:47.617141 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:47.617489 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:47.619347 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:47.620659 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:47.622529 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:47.623266 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:47.623423 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:47.628327 (+ 4904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:47.628339 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:47.628360 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:47.628414 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:47.632565 (+ 4151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:47.633576 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:47.633583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:47.633589 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:47.633816 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:47.634325 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:47.634331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:47.634334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:47.635009 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:47.635329 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:47.635674 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 03:11:47.636117 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":394,"threads_started":1}
I20260811 03:11:47.644737 17777 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68e5481330445e88c4a9f45d61d282" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35403 } }
I20260811 03:11:47.645192 17777 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:47.645337 17777 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb68e5481330445e88c4a9f45d61d282, State: Initialized, Role: FOLLOWER
I20260811 03:11:47.645900 17777 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68e5481330445e88c4a9f45d61d282" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35403 } }
I20260811 03:11:47.646137 17777 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:47.646289 17777 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:47.646510 17777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:47.648712 17777 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68e5481330445e88c4a9f45d61d282" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35403 } }
I20260811 03:11:47.649204 17777 leader_election.cc:302] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb68e5481330445e88c4a9f45d61d282; no voters:
I20260811 03:11:47.649873 17777 leader_election.cc:288] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:47.650009 17784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:47.651060 17784 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [term 1 LEADER]: Becoming Leader. State: Replica: eb68e5481330445e88c4a9f45d61d282, State: Running, Role: LEADER
I20260811 03:11:47.653131 17784 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68e5481330445e88c4a9f45d61d282" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35403 } }
I20260811 03:11:47.653450 17777 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:47.655954 17786 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb68e5481330445e88c4a9f45d61d282. Latest consensus state: current_term: 1 leader_uuid: "eb68e5481330445e88c4a9f45d61d282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68e5481330445e88c4a9f45d61d282" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35403 } } }
I20260811 03:11:47.656335 17786 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:47.657173 17785 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb68e5481330445e88c4a9f45d61d282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68e5481330445e88c4a9f45d61d282" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35403 } } }
I20260811 03:11:47.657655 17785 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:47.658670 17790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:47.665004 17790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:47.674027 17790 catalog_manager.cc:1347] Generated new cluster ID: c4b5c82b99824a1397ac32b5c01ba625
I20260811 03:11:47.674147 17790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:47.686568 17790 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:47.688457 17790 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:47.697019 17790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb68e5481330445e88c4a9f45d61d282: Generated new TSK 0
I20260811 03:11:47.698161 17790 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:11:47.766268 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:35403
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:48.110266 17808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:48.110579 17808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:48.110672 17808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:48.117686 17808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:48.117787 17808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:48.117897 17808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:48.128717 17808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:35403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:48.130009 17808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:48.131453 17808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:48.144886 17816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:48.144881 17814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:48.144922 17813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:48.145869 17808 server_base.cc:1034] running on GCE node
I20260811 03:11:48.146523 17808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:48.147439 17808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:48.148669 17808 hybrid_clock.cc:648] HybridClock initialized: now 1786417908148621 us; error 33 us; skew 500 ppm
I20260811 03:11:48.148976 17808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:48 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417908, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:48.156170 17808 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:11:48.157586 17808 webserver.cc:460] Webserver started at http://127.11.60.1:40163/ using document root <none> and password file <none>
I20260811 03:11:48.158380 17808 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:48.158499 17808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:48.158870 17808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:48.162004 17808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "85ad1b8af8a649d4b6ef14d3145d0829"
format_stamp: "Formatted at 2026-08-11 03:11:48 on dist-test-slave-tm1g"
server_key: "0be955363f9df0b2114a0d7045114d4e"
server_key_iv: "75c20db37e360de19af83ca914a16d3a"
server_key_version: "encryptionkey@0"
I20260811 03:11:48.162866 17808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "85ad1b8af8a649d4b6ef14d3145d0829"
format_stamp: "Formatted at 2026-08-11 03:11:48 on dist-test-slave-tm1g"
server_key: "0be955363f9df0b2114a0d7045114d4e"
server_key_iv: "75c20db37e360de19af83ca914a16d3a"
server_key_version: "encryptionkey@0"
I20260811 03:11:48.168454 17808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:11:48.172209 17823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:48.173671 17808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:11:48.173874 17808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "85ad1b8af8a649d4b6ef14d3145d0829"
format_stamp: "Formatted at 2026-08-11 03:11:48 on dist-test-slave-tm1g"
server_key: "0be955363f9df0b2114a0d7045114d4e"
server_key_iv: "75c20db37e360de19af83ca914a16d3a"
server_key_version: "encryptionkey@0"
I20260811 03:11:48.174108 17808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:48.197988 17808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:48.202832 17808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:48.203145 17808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:48.204169 17808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:48.205570 17808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:48.205679 17808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:48.205842 17808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:48.205919 17808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:48.243440 17808 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:36071
I20260811 03:11:48.243541 17946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:36071 every 8 connection(s)
I20260811 03:11:48.244899 17808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:48.251875 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 17808
I20260811 03:11:48.252104 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:48.252966 11504 external_mini_cluster.cc:1428] Setting key 21c37f1c15b7da983b60275a6f3b6764
Aug 11 03:11:48 dist-test-slave-tm1g krb5kdc[17686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417908, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:48.269989 17950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:48.247761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:37109 (local address 127.11.60.62:35403)
0811 03:11:48.248223 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:48.248228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:48.249337 (+ 1109us) server_negotiation.cc:407] Connection header received
0811 03:11:48.251653 (+ 2316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:48.251658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:48.251737 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:48.251851 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:48.254011 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:48.255076 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:48.256768 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:48.257536 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:48.257682 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:48.261916 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:48.261921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:48.261925 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:48.261959 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:48.264984 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:48.266188 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:48.266196 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:48.266202 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:48.266446 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:48.267541 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:48.267548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:48.267553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:48.268032 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:48.268234 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:48.269167 (+ 933us) server_negotiation.cc:299] Negotiation successful
0811 03:11:48.269469 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":234,"threads_started":1}
I20260811 03:11:48.270720 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:48.248352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:35403 (local address 127.11.60.1:37109)
0811 03:11:48.249162 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:48.249211 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:48.251304 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:48.252136 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:48.252151 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:48.252991 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:48.253828 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:48.253862 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:48.255222 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:48.255227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:48.256615 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:48.256627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:48.257750 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:48.257755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:48.257948 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:48.258685 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:48.258709 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:48.261690 (+ 2981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:48.265153 (+ 3463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:48.265160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:48.265173 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:48.265938 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:48.266908 (+ 970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:48.266914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:48.266923 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:48.267240 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:48.268286 (+ 1046us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:48.268302 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:48.268701 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 03:11:48.268979 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 03:11:48.269436 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":337,"threads_started":1}
I20260811 03:11:48.272378 17947 heartbeater.cc:347] Connected to a master server at 127.11.60.62:35403
I20260811 03:11:48.272723 17947 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:48.273669 17947 heartbeater.cc:511] Master 127.11.60.62:35403 requested a full tablet report, sending...
I20260811 03:11:48.276135 17735 ts_manager.cc:194] Registered new tserver with Master: 85ad1b8af8a649d4b6ef14d3145d0829 (127.11.60.1:36071)
I20260811 03:11:48.277794 17735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:37109
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:11:48.317909 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:35403
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:48.663743 17955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:48.664036 17955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:48.664135 17955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:48.671291 17955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:48.671402 17955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:48.671588 17955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:48.682478 17955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:35403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:48.683881 17955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:48.685302 17955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:48.699909 17960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:48.699878 17961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:48.699944 17963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:48.701555 17955 server_base.cc:1034] running on GCE node
I20260811 03:11:48.702178 17955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:48.703178 17955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:48.704411 17955 hybrid_clock.cc:648] HybridClock initialized: now 1786417908704370 us; error 83 us; skew 500 ppm
I20260811 03:11:48.704732 17955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:48 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417908, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:48.708256 17955 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:11:48.709873 17955 webserver.cc:460] Webserver started at http://127.11.60.2:42241/ using document root <none> and password file <none>
I20260811 03:11:48.710687 17955 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:48.710841 17955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:48.711169 17955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:48.714219 17955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "bc5fe3f7d03a4d4d845b2eab84293465"
format_stamp: "Formatted at 2026-08-11 03:11:48 on dist-test-slave-tm1g"
server_key: "bf5d1aefdcf705d5dd64df5ddaeeec88"
server_key_iv: "61763911d3d42839a45480118477e8f2"
server_key_version: "encryptionkey@0"
I20260811 03:11:48.715163 17955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "bc5fe3f7d03a4d4d845b2eab84293465"
format_stamp: "Formatted at 2026-08-11 03:11:48 on dist-test-slave-tm1g"
server_key: "bf5d1aefdcf705d5dd64df5ddaeeec88"
server_key_iv: "61763911d3d42839a45480118477e8f2"
server_key_version: "encryptionkey@0"
I20260811 03:11:48.720585 17955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:11:48.724457 17970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:48.725872 17955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 03:11:48.726056 17955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "bc5fe3f7d03a4d4d845b2eab84293465"
format_stamp: "Formatted at 2026-08-11 03:11:48 on dist-test-slave-tm1g"
server_key: "bf5d1aefdcf705d5dd64df5ddaeeec88"
server_key_iv: "61763911d3d42839a45480118477e8f2"
server_key_version: "encryptionkey@0"
I20260811 03:11:48.726275 17955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:48.771315 17955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:48.775707 17955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:48.776015 17955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:48.777024 17955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:48.778503 17955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:48.778607 17955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:48.778759 17955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:48.778832 17955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:48.813496 17955 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:46757
I20260811 03:11:48.813501 18089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:46757 every 8 connection(s)
I20260811 03:11:48.814926 17955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:48.824055 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 17955
I20260811 03:11:48.824242 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:48.824741 11504 external_mini_cluster.cc:1428] Setting key 957730c5f6dd2ffff74ef577f0c4c6a2
Aug 11 03:11:48 dist-test-slave-tm1g krb5kdc[17686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417908, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:48.837064 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:48.817683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:45019 (local address 127.11.60.62:35403)
0811 03:11:48.818184 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:48.818189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:48.819298 (+ 1109us) server_negotiation.cc:407] Connection header received
0811 03:11:48.821278 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:48.821284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:48.821387 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:48.821521 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:48.823176 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:48.823930 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:48.825822 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:48.826369 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:48.826512 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:48.831086 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:48.831092 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:48.831095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:48.831131 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:48.833783 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:48.834649 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:48.834653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:48.834655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:48.834778 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:48.835250 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:48.835255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:48.835258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:48.835556 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:48.835714 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:48.836449 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 03:11:48.836679 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":270,"threads_started":1}
I20260811 03:11:48.838042 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:48.818256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:35403 (local address 127.11.60.2:45019)
0811 03:11:48.819111 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:48.819159 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:48.820938 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:48.821772 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:48.821781 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:48.822401 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:48.823000 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:48.823017 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:48.824085 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:48.824094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:48.825679 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:48.825693 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:48.826607 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:48.826615 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:48.826910 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:48.827714 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:48.827758 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:48.830818 (+ 3060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:48.833963 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:48.833972 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:48.833993 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:48.834510 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:48.834904 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:48.834909 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:48.834913 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:48.835119 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:48.835703 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:48.835711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:48.836067 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 03:11:48.836343 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 03:11:48.836919 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":338,"threads_started":1}
I20260811 03:11:48.839605 18090 heartbeater.cc:347] Connected to a master server at 127.11.60.62:35403
I20260811 03:11:48.839936 18090 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:48.840658 18090 heartbeater.cc:511] Master 127.11.60.62:35403 requested a full tablet report, sending...
I20260811 03:11:48.842238 17735 ts_manager.cc:194] Registered new tserver with Master: bc5fe3f7d03a4d4d845b2eab84293465 (127.11.60.2:46757)
I20260811 03:11:48.843124 17735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:45019
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:11:48.882709 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:35403
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:49.218713 18097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:49.219023 18097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:49.219120 18097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:49.227427 18097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:49.227552 18097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:49.227669 18097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:49.237900 18097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:35403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:49.239245 18097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:49.240533 18097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:49.255383 18103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:49.255483 18106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:49.255658 18102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:49.256623 18097 server_base.cc:1034] running on GCE node
I20260811 03:11:49.257252 18097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:49.258226 18097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:49.259501 18097 hybrid_clock.cc:648] HybridClock initialized: now 1786417909259454 us; error 62 us; skew 500 ppm
I20260811 03:11:49.259898 18097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:49.263237 18097 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:11:49.264742 18097 webserver.cc:460] Webserver started at http://127.11.60.3:42241/ using document root <none> and password file <none>
I20260811 03:11:49.265463 18097 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:49.265619 18097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:49.265937 18097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:49.269011 18097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "11cd4ddea3ed4b40b8b69e231e48a387"
format_stamp: "Formatted at 2026-08-11 03:11:49 on dist-test-slave-tm1g"
server_key: "2eca787cb54e1daf6da4dce12915be33"
server_key_iv: "df708e8d6b9df084c8c059e256deb888"
server_key_version: "encryptionkey@0"
I20260811 03:11:49.269810 18097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "11cd4ddea3ed4b40b8b69e231e48a387"
format_stamp: "Formatted at 2026-08-11 03:11:49 on dist-test-slave-tm1g"
server_key: "2eca787cb54e1daf6da4dce12915be33"
server_key_iv: "df708e8d6b9df084c8c059e256deb888"
server_key_version: "encryptionkey@0"
I20260811 03:11:49.275167 18097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 03:11:49.278752 18114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:49.280072 18097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 03:11:49.280248 18097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "11cd4ddea3ed4b40b8b69e231e48a387"
format_stamp: "Formatted at 2026-08-11 03:11:49 on dist-test-slave-tm1g"
server_key: "2eca787cb54e1daf6da4dce12915be33"
server_key_iv: "df708e8d6b9df084c8c059e256deb888"
server_key_version: "encryptionkey@0"
I20260811 03:11:49.280467 18097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:49.281898 17947 heartbeater.cc:503] Master 127.11.60.62:35403 was elected leader, sending a full tablet report...
I20260811 03:11:49.304945 18097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:49.309267 18097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:49.309609 18097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:49.310684 18097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:49.313163 18097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:49.313259 18097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:49.313382 18097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:49.313453 18097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:49.348767 18097 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:40975
I20260811 03:11:49.348861 18236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:40975 every 8 connection(s)
I20260811 03:11:49.350139 18097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:49.357635 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 18097
I20260811 03:11:49.357803 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:49.358327 11504 external_mini_cluster.cc:1428] Setting key 04e052569f643785478ef6cb033f9419
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:11:49.371486 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:49.352822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:51705 (local address 127.11.60.62:35403)
0811 03:11:49.353369 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:49.353377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:49.354639 (+ 1262us) server_negotiation.cc:407] Connection header received
0811 03:11:49.356433 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:49.356437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:49.356525 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:49.356640 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:49.358378 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.359482 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.361475 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.361961 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.362067 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:49.365739 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:49.365744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:49.365747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:49.365782 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:49.368557 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.369247 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.369251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.369253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.369371 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.369787 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.369791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.369793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.370114 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:49.370242 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:49.370984 (+ 742us) server_negotiation.cc:299] Negotiation successful
0811 03:11:49.371161 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":260,"threads_started":1}
I20260811 03:11:49.372529 18240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:49.353412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:35403 (local address 127.11.60.3:51705)
0811 03:11:49.354462 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:49.354519 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:49.356164 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:49.356869 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:49.356878 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:49.357408 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:49.358166 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:49.358186 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.359655 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.359659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:49.361331 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:49.361345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.362177 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.362186 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:49.362522 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:49.363179 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:49.363201 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:49.365500 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:49.368680 (+ 3180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:49.368687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:49.368700 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:49.369139 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:49.369466 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:49.369469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:49.369471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:49.369645 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:49.370247 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:49.370253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:49.370638 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 03:11:49.370878 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 03:11:49.371280 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":335,"threads_started":1}
I20260811 03:11:49.374046 18237 heartbeater.cc:347] Connected to a master server at 127.11.60.62:35403
I20260811 03:11:49.374451 18237 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:49.375152 18237 heartbeater.cc:511] Master 127.11.60.62:35403 requested a full tablet report, sending...
I20260811 03:11:49.376606 17735 ts_manager.cc:194] Registered new tserver with Master: 11cd4ddea3ed4b40b8b69e231e48a387 (127.11.60.3:40975)
I20260811 03:11:49.377490 17735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:51705
I20260811 03:11:49.386667 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/test-admin.keytab.
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:49.418211 11504 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 03:11:49.423933 18245 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:11:49.421077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53772 (local address 127.11.60.1:36071)
0811 03:11:49.421703 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:49.421711 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:49.421761 (+ 50us) server_negotiation.cc:407] Connection header received
0811 03:11:49.421823 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:49.421831 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:49.422089 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:49.422321 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:49.423413 (+ 1092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53772: BlockingRecv error: recv got EOF from 127.0.0.1:53772 (error 108)
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":313,"threads_started":1}
W20260811 03:11:49.424013 11504 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5cc' of type FILE
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:49.425730 11504 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:49.427161 11504 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:49.442973 18245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:49.428991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.11.60.1:36071)
0811 03:11:49.429174 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:49.429179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:49.429255 (+ 76us) server_negotiation.cc:407] Connection header received
0811 03:11:49.429496 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:49.429500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:49.429577 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:49.429658 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:49.430875 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.431899 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.433181 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.433751 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.433847 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:49.437570 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:49.437581 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:49.437597 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:49.437631 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:49.440451 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.441059 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.441065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.441068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.441180 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.441477 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.441480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.441482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.441839 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:49.442059 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:49.442391 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 03:11:49.442676 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 03:11:49.448700 18245 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:11:49.447162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.11.60.1:36071)
0811 03:11:49.447378 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:49.447382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:49.447482 (+ 100us) server_negotiation.cc:407] Connection header received
0811 03:11:49.447623 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:49.447626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:49.447680 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:49.447756 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:49.448449 (+ 693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53798: BlockingRecv error: recv got EOF from 127.0.0.1:53798 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 03:11:49.448820 11504 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5cc' of type FILE
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:49.450544 11504 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:49.451807 11504 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:49.466713 18245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:49.453253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53810 (local address 127.11.60.1:36071)
0811 03:11:49.453492 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:49.453498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:49.453752 (+ 254us) server_negotiation.cc:407] Connection header received
0811 03:11:49.453877 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:49.453881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:49.453951 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:49.454031 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:49.455182 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.455885 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.457068 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.457501 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.457599 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:49.461247 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:49.461252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:49.461256 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:49.461297 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:49.464110 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.464742 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.464746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.464748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.464869 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.465273 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.465278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.465282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.465578 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:49.465874 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:49.466162 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 03:11:49.466412 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260811 03:11:49.471321 18245 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:11:49.469650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.11.60.1:36071)
0811 03:11:49.469827 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:49.469832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:49.469916 (+ 84us) server_negotiation.cc:407] Connection header received
0811 03:11:49.470049 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:49.470052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:49.470121 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:49.470208 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:49.471035 (+ 827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53814: BlockingRecv error: recv got EOF from 127.0.0.1:53814 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 03:11:49.471560 11504 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417858274425-11504-0/krb5kdc/krb5cc' of type FILE
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:49.473083 11504 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:49.474437 11504 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:11:49.489624 18245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:49.475834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53828 (local address 127.11.60.1:36071)
0811 03:11:49.476078 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:49.476085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:49.476345 (+ 260us) server_negotiation.cc:407] Connection header received
0811 03:11:49.476469 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:49.476473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:49.476531 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:49.476614 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:49.477711 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.478558 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.479752 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.480159 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.480251 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:49.483970 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:49.483974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:49.483978 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:49.484012 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:49.486924 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.487643 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.487647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.487649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.487828 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.488191 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.488195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.488197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.488525 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:49.488747 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:49.489039 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 03:11:49.489290 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 03:11:49.491570 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 17808
I20260811 03:11:49.504424 18245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:49.492329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53836 (local address 127.11.60.1:36071)
0811 03:11:49.492502 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:49.492506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:49.492940 (+ 434us) server_negotiation.cc:407] Connection header received
0811 03:11:49.493125 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:49.493129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:49.493188 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:49.493281 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:49.494257 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.495099 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.496260 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.496686 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.496780 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:49.499023 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:49.499028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:49.499032 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:49.499062 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:49.501727 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.502526 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.502530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.502533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.502682 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.503077 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.503081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.503083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.503382 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:49.503619 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:49.503984 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 03:11:49.504166 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 03:11:49.505098 17942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:49.590201 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 17808
I20260811 03:11:49.610180 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 17955
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:11:49.625741 18262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:49.611203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.11.60.2:46757)
0811 03:11:49.611749 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:49.611757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:49.611782 (+ 25us) server_negotiation.cc:407] Connection header received
0811 03:11:49.611841 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:49.611847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:49.612063 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:49.612250 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:49.613263 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.614366 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.615632 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.616197 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.616335 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:49.620213 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:49.620224 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:49.620239 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:49.620289 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:49.623221 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.623867 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.623874 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.623876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.624005 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.624332 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.624335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.624337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.624692 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:49.624902 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:49.625195 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 03:11:49.625410 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":291,"threads_started":1}
I20260811 03:11:49.627267 18084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:49.712143 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 17955
I20260811 03:11:49.730815 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 18097
Aug 11 03:11:49 dist-test-slave-tm1g krb5kdc[17686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:11:49.749233 18264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:49.731738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.11.60.3:40975)
0811 03:11:49.732319 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:49.732326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:49.732354 (+ 28us) server_negotiation.cc:407] Connection header received
0811 03:11:49.732417 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:49.732423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:49.732652 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:49.732828 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:49.733933 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.735024 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.736243 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:49.736805 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:49.736945 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:49.741821 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:49.741835 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:49.741853 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:49.741923 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:49.746363 (+ 4440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.747060 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.747070 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.747073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.747211 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:49.747611 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:49.747617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:49.747620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:49.747979 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:49.748315 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:49.748622 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 03:11:49.748891 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":281,"threads_started":1}
I20260811 03:11:49.750617 18232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:49.834199 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 18097
I20260811 03:11:49.853629 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 17699
I20260811 03:11:49.855060 17772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:49.963938 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 17699
2026-08-11T03:11:49Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4243 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:11:50 dist-test-slave-tm1g krb5kdc[18269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:11:50 dist-test-slave-tm1g krb5kdc[18269](info): set up 2 sockets
Aug 11 03:11:50 dist-test-slave-tm1g krb5kdc[18269](info): commencing operation
krb5kdc: starting...
W20260811 03:11:51.133801 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:11:51 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:11:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:11:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.62.keytab.
Entry for principal oryx/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.62.keytab.
I20260811 03:11:51.403015 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42751
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.62.keytab
--principal=oryx/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:42751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:51.756712 18282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:51.757004 18282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:51.757094 18282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:51.764400 18282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:51.764528 18282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:51.764577 18282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:51.764626 18282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:51.764673 18282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:51.776365 18282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:42751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.62.keytab
--principal=oryx/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:51.777948 18282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:51.779428 18282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:51.792987 18287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:51.792953 18288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:51.793447 18290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:51.793815 18282 server_base.cc:1034] running on GCE node
I20260811 03:11:51.794816 18282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:51.796432 18282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:51.797712 18282 hybrid_clock.cc:648] HybridClock initialized: now 1786417911797679 us; error 66 us; skew 500 ppm
I20260811 03:11:51.798105 18282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:51 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417911, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:51.801478 18282 init.cc:377] Logged in from keytab as oryx/127.11.60.62@KRBTEST.COM (short username oryx)
I20260811 03:11:51.803186 18282 webserver.cc:460] Webserver started at http://127.11.60.62:33193/ using document root <none> and password file <none>
I20260811 03:11:51.803916 18282 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:51.804080 18282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:51.804404 18282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:51.807672 18282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "332214273aae4803b3f58ceca1e2b794"
format_stamp: "Formatted at 2026-08-11 03:11:51 on dist-test-slave-tm1g"
server_key: "17e5578cb47d626e26050bd570609ad8"
server_key_iv: "939b66cc7ced55e2d135b16300029c81"
server_key_version: "encryptionkey@0"
I20260811 03:11:51.808501 18282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "332214273aae4803b3f58ceca1e2b794"
format_stamp: "Formatted at 2026-08-11 03:11:51 on dist-test-slave-tm1g"
server_key: "17e5578cb47d626e26050bd570609ad8"
server_key_iv: "939b66cc7ced55e2d135b16300029c81"
server_key_version: "encryptionkey@0"
I20260811 03:11:51.814208 18282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 03:11:51.818109 18297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:51.819615 18282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 03:11:51.819823 18282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "332214273aae4803b3f58ceca1e2b794"
format_stamp: "Formatted at 2026-08-11 03:11:51 on dist-test-slave-tm1g"
server_key: "17e5578cb47d626e26050bd570609ad8"
server_key_iv: "939b66cc7ced55e2d135b16300029c81"
server_key_version: "encryptionkey@0"
I20260811 03:11:51.820046 18282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:51.831817 18282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:51.835896 18282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:51.836185 18282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:51.855391 18282 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:42751
I20260811 03:11:51.855427 18355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:42751 every 8 connection(s)
I20260811 03:11:51.856907 18282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:11:51.858878 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 18282
I20260811 03:11:51.859087 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:11:51.859745 11504 external_mini_cluster.cc:1428] Setting key 3dcf7da69e5748440c2f21ff5a4ab0f2
I20260811 03:11:51.861250 18356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 03:11:51 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:11:51.875941 18356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794: Bootstrap starting.
I20260811 03:11:51.879546 18356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:51.880667 18356 log.cc:824] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:51.883818 18356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794: No bootstrap required, opened a new log
I20260811 03:11:51.884426 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:51.861854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57296 (local address 127.11.60.62:42751)
0811 03:11:51.862701 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:51.862713 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:51.862753 (+ 40us) server_negotiation.cc:407] Connection header received
0811 03:11:51.864285 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:51.864308 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:51.864683 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:51.865016 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:51.866913 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:51.868290 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:51.870087 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:51.870987 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:51.871139 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:51.876218 (+ 5079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:51.876227 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:51.876243 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:51.876290 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:51.879793 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:51.880620 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:51.880631 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:51.880645 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:51.880870 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:51.881313 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:51.881319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:51.881324 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:51.881957 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:51.882339 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:51.882855 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 03:11:51.883315 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":404,"threads_started":1}
I20260811 03:11:51.892354 18356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } }
I20260811 03:11:51.892757 18356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:51.892905 18356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 332214273aae4803b3f58ceca1e2b794, State: Initialized, Role: FOLLOWER
I20260811 03:11:51.893543 18356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } }
I20260811 03:11:51.893806 18356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:11:51.893957 18356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:11:51.894181 18356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:51.896826 18356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } }
I20260811 03:11:51.897337 18356 leader_election.cc:302] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 332214273aae4803b3f58ceca1e2b794; no voters:
I20260811 03:11:51.898087 18356 leader_election.cc:288] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:11:51.898222 18361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:51.899693 18361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 1 LEADER]: Becoming Leader. State: Replica: 332214273aae4803b3f58ceca1e2b794, State: Running, Role: LEADER
I20260811 03:11:51.900270 18361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } }
I20260811 03:11:51.901110 18356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:51.904505 18362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 332214273aae4803b3f58ceca1e2b794. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } }
I20260811 03:11:51.905429 18362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:51.905289 18363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } }
I20260811 03:11:51.905633 18363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:51.906716 18369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:11:51.917730 18369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:11:51.930320 18369 catalog_manager.cc:1347] Generated new cluster ID: 9d125812b9264ea99502cb768c149147
I20260811 03:11:51.930457 18369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:11:51.930932 18379 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 03:11:51.944761 18369 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:11:51.946849 18369 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:11:51.965445 18369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794: Generated new TSK 0
I20260811 03:11:51.966406 18369 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab.
Entry for principal oryx/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab.
I20260811 03:11:52.047994 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42751
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab
--principal=oryx/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:52.399786 18384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:52.400158 18384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:52.400339 18384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:52.408131 18384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:52.408289 18384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:52.408463 18384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:52.418838 18384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab
--principal=oryx/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:52.420292 18384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:52.421696 18384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:52.436436 18389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:52.436475 18392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:52.436501 18390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:52.437428 18384 server_base.cc:1034] running on GCE node
I20260811 03:11:52.438184 18384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:52.439386 18384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:52.440637 18384 hybrid_clock.cc:648] HybridClock initialized: now 1786417912440582 us; error 48 us; skew 500 ppm
I20260811 03:11:52.440992 18384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:52 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417912, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:52.444453 18384 init.cc:377] Logged in from keytab as oryx/127.11.60.1@KRBTEST.COM (short username oryx)
I20260811 03:11:52.446132 18384 webserver.cc:460] Webserver started at http://127.11.60.1:40515/ using document root <none> and password file <none>
I20260811 03:11:52.446957 18384 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:52.447103 18384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:52.447445 18384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:52.450637 18384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "9cfcb8537b49453096cea3274246b3d2"
format_stamp: "Formatted at 2026-08-11 03:11:52 on dist-test-slave-tm1g"
server_key: "00ed5f9546a70176c291f0d09349361d"
server_key_iv: "49169756f7f810e19816099ecf12abf8"
server_key_version: "encryptionkey@0"
I20260811 03:11:52.451511 18384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "9cfcb8537b49453096cea3274246b3d2"
format_stamp: "Formatted at 2026-08-11 03:11:52 on dist-test-slave-tm1g"
server_key: "00ed5f9546a70176c291f0d09349361d"
server_key_iv: "49169756f7f810e19816099ecf12abf8"
server_key_version: "encryptionkey@0"
I20260811 03:11:52.457306 18384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:11:52.461211 18399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:52.462940 18384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:52.463142 18384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "9cfcb8537b49453096cea3274246b3d2"
format_stamp: "Formatted at 2026-08-11 03:11:52 on dist-test-slave-tm1g"
server_key: "00ed5f9546a70176c291f0d09349361d"
server_key_iv: "49169756f7f810e19816099ecf12abf8"
server_key_version: "encryptionkey@0"
I20260811 03:11:52.463385 18384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:52.481154 18384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:52.485908 18384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:52.486227 18384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:52.487278 18384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:52.488754 18384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:52.488862 18384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:52.489029 18384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:52.489106 18384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:52.525219 18384 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:39959
I20260811 03:11:52.525259 18520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:39959 every 8 connection(s)
I20260811 03:11:52.526655 18384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:52.536021 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 18384
I20260811 03:11:52.536259 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:11:52.536981 11504 external_mini_cluster.cc:1428] Setting key 2ac775bf6c8d2b5ce8bbdafab9631c37
Aug 11 03:11:52 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417912, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:11:52.549659 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:52.529528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:45753 (local address 127.11.60.62:42751)
0811 03:11:52.530046 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:52.530050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:52.530944 (+ 894us) server_negotiation.cc:407] Connection header received
0811 03:11:52.532849 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:52.532855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:52.532962 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:52.533096 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:52.534731 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:52.535470 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:52.537231 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:52.537807 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:52.537978 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:52.542370 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:52.542380 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:52.542387 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:52.542438 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:52.546275 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:52.547140 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:52.547148 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:52.547154 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:52.547316 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:52.547751 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:52.547758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:52.547763 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:52.548168 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:52.548328 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:52.549079 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 03:11:52.549376 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":288,"threads_started":1}
I20260811 03:11:52.551015 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:52.529982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.1:45753)
0811 03:11:52.530771 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:52.530816 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:52.532559 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:52.533352 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:52.533361 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:52.533888 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:52.534548 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:52.534568 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:52.535650 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:52.535654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:52.537012 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:52.537025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:52.538074 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:52.538079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:52.538405 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:52.539298 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:52.539338 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:52.542104 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:52.546482 (+ 4378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:52.546490 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:52.546504 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:52.546991 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:52.547429 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:52.547433 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:52.547435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:52.547630 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:52.548292 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:52.548299 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:52.548670 (+ 371us) client_negotiation.cc:769] Sending connection context
0811 03:11:52.548907 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 03:11:52.549401 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":357,"threads_started":1}
I20260811 03:11:52.552874 18521 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42751
I20260811 03:11:52.553200 18521 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:52.553992 18521 heartbeater.cc:511] Master 127.11.60.62:42751 requested a full tablet report, sending...
I20260811 03:11:52.556558 18315 ts_manager.cc:194] Registered new tserver with Master: 9cfcb8537b49453096cea3274246b3d2 (127.11.60.1:39959)
I20260811 03:11:52.558826 18315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:45753
WARNING: no policy specified for oryx/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab.
Entry for principal oryx/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab.
I20260811 03:11:52.598989 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42751
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab
--principal=oryx/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:52.945801 18527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:52.946118 18527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:52.946218 18527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:52.953747 18527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:52.953877 18527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:52.953996 18527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:52.964345 18527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab
--principal=oryx/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:52.965791 18527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:52.967224 18527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:52.981595 18532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:52.981662 18535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:52.981621 18533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:52.982530 18527 server_base.cc:1034] running on GCE node
I20260811 03:11:52.983211 18527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:52.984299 18527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:52.985540 18527 hybrid_clock.cc:648] HybridClock initialized: now 1786417912985491 us; error 38 us; skew 500 ppm
I20260811 03:11:52.985906 18527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:52 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417912, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:52.989475 18527 init.cc:377] Logged in from keytab as oryx/127.11.60.2@KRBTEST.COM (short username oryx)
I20260811 03:11:52.991299 18527 webserver.cc:460] Webserver started at http://127.11.60.2:36309/ using document root <none> and password file <none>
I20260811 03:11:52.992112 18527 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:52.992274 18527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:52.992605 18527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:52.995962 18527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "ee8e84046be747fc98b50003173b3342"
format_stamp: "Formatted at 2026-08-11 03:11:52 on dist-test-slave-tm1g"
server_key: "87ad67f7816d7901e7082e383467fe42"
server_key_iv: "b775c7fab26fa860602fdc60beeb4d59"
server_key_version: "encryptionkey@0"
I20260811 03:11:52.996822 18527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "ee8e84046be747fc98b50003173b3342"
format_stamp: "Formatted at 2026-08-11 03:11:52 on dist-test-slave-tm1g"
server_key: "87ad67f7816d7901e7082e383467fe42"
server_key_iv: "b775c7fab26fa860602fdc60beeb4d59"
server_key_version: "encryptionkey@0"
I20260811 03:11:53.002808 18527 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 03:11:53.006728 18544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:53.008391 18527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:11:53.008697 18527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "ee8e84046be747fc98b50003173b3342"
format_stamp: "Formatted at 2026-08-11 03:11:52 on dist-test-slave-tm1g"
server_key: "87ad67f7816d7901e7082e383467fe42"
server_key_iv: "b775c7fab26fa860602fdc60beeb4d59"
server_key_version: "encryptionkey@0"
I20260811 03:11:53.008929 18527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:53.026892 18527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:53.031651 18527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:53.031965 18527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:53.033007 18527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:53.034533 18527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:53.034662 18527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:53.034818 18527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:53.034869 18527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:53.076292 18527 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:43797
I20260811 03:11:53.076327 18672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:43797 every 8 connection(s)
I20260811 03:11:53.078189 18527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:53.086274 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 18527
I20260811 03:11:53.086800 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:11:53.087348 11504 external_mini_cluster.cc:1428] Setting key ad874dddab47532bcd2204121e4dd468
Aug 11 03:11:53 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417912, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:11:53.105355 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:53.081368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:57551 (local address 127.11.60.62:42751)
0811 03:11:53.081897 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:53.081905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:53.083020 (+ 1115us) server_negotiation.cc:407] Connection header received
0811 03:11:53.085291 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:53.085297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:53.085392 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:53.085505 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:53.088185 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:53.089185 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:53.091723 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:53.092567 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:53.092741 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:53.097070 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:53.097079 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:53.097087 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:53.097136 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:53.101306 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:53.102559 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:53.102565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:53.102569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:53.102729 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:53.103314 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:53.103318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:53.103320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:53.103694 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:53.103825 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:53.104846 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0811 03:11:53.105047 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":244,"threads_started":1}
I20260811 03:11:53.107079 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:53.081659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.2:57551)
0811 03:11:53.082660 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:53.082873 (+ 213us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:53.084956 (+ 2083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:53.085826 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:53.085835 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:53.087194 (+ 1359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:53.087947 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:53.087972 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:53.089518 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:53.089524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:53.091358 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:53.091372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:53.092854 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:53.092860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:53.093107 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:53.093924 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:53.093950 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:53.096767 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:53.101531 (+ 4764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:53.101544 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:53.101567 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:53.102382 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:53.102854 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:53.102862 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:53.102868 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:53.103151 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:53.103839 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:53.103850 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:53.104375 (+ 525us) client_negotiation.cc:769] Sending connection context
0811 03:11:53.104738 (+ 363us) client_negotiation.cc:240] Negotiation successful
0811 03:11:53.105413 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":388,"threads_started":1}
I20260811 03:11:53.109093 18673 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42751
I20260811 03:11:53.109458 18673 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:53.110455 18673 heartbeater.cc:511] Master 127.11.60.62:42751 requested a full tablet report, sending...
I20260811 03:11:53.112557 18315 ts_manager.cc:194] Registered new tserver with Master: ee8e84046be747fc98b50003173b3342 (127.11.60.2:43797)
I20260811 03:11:53.113927 18315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:57551
WARNING: no policy specified for oryx/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab.
Entry for principal oryx/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab.
I20260811 03:11:53.154295 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42751
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab
--principal=oryx/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:53.510870 18681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:53.511227 18681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:53.511404 18681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:53.519220 18681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:53.519380 18681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:53.519599 18681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:53.529963 18681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab
--principal=oryx/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:53.531447 18681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:53.532757 18681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:53.546554 18687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:53.546690 18691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:53.546824 18681 server_base.cc:1034] running on GCE node
W20260811 03:11:53.546718 18689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:53.547760 18681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:53.548749 18681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:53.549974 18681 hybrid_clock.cc:648] HybridClock initialized: now 1786417913549939 us; error 35 us; skew 500 ppm
I20260811 03:11:53.550407 18681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:53 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417913, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:53.557451 18681 init.cc:377] Logged in from keytab as oryx/127.11.60.3@KRBTEST.COM (short username oryx)
I20260811 03:11:53.559052 18681 webserver.cc:460] Webserver started at http://127.11.60.3:33445/ using document root <none> and password file <none>
I20260811 03:11:53.559788 18681 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:53.559966 18681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:53.560292 18681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:53.562285 18521 heartbeater.cc:503] Master 127.11.60.62:42751 was elected leader, sending a full tablet report...
I20260811 03:11:53.563910 18681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "884a718d1b9b4654a00529bffe0ba706"
format_stamp: "Formatted at 2026-08-11 03:11:53 on dist-test-slave-tm1g"
server_key: "100266dd8994ea1eefd004a1760e276e"
server_key_iv: "ff074425a326ebc07ec5de5f4a28d38b"
server_key_version: "encryptionkey@0"
I20260811 03:11:53.564903 18681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "884a718d1b9b4654a00529bffe0ba706"
format_stamp: "Formatted at 2026-08-11 03:11:53 on dist-test-slave-tm1g"
server_key: "100266dd8994ea1eefd004a1760e276e"
server_key_iv: "ff074425a326ebc07ec5de5f4a28d38b"
server_key_version: "encryptionkey@0"
I20260811 03:11:53.570833 18681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 03:11:53.574832 18702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:53.576241 18681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:11:53.576436 18681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "884a718d1b9b4654a00529bffe0ba706"
format_stamp: "Formatted at 2026-08-11 03:11:53 on dist-test-slave-tm1g"
server_key: "100266dd8994ea1eefd004a1760e276e"
server_key_iv: "ff074425a326ebc07ec5de5f4a28d38b"
server_key_version: "encryptionkey@0"
I20260811 03:11:53.576673 18681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:53.598237 18681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:53.603137 18681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:53.603464 18681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:53.604497 18681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:53.605986 18681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:53.606091 18681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:53.606241 18681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:53.606331 18681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:53.642009 18681 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:42001
I20260811 03:11:53.642099 18825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:42001 every 8 connection(s)
I20260811 03:11:53.643357 18681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:53.652141 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 18681
I20260811 03:11:53.652319 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:11:53.652781 11504 external_mini_cluster.cc:1428] Setting key 3a284cf7a3bec034c5fa2e8b5c240d44
Aug 11 03:11:53 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417913, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:11:53.663538 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:53.645908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:54551 (local address 127.11.60.62:42751)
0811 03:11:53.646431 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:53.646435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:53.647227 (+ 792us) server_negotiation.cc:407] Connection header received
0811 03:11:53.649133 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:53.649138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:53.649231 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:53.649350 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:53.650892 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:53.651774 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:53.653594 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:53.654147 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:53.654297 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:53.657853 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:53.657858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:53.657861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:53.657899 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:53.660383 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:53.661120 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:53.661132 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:53.661135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:53.661262 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:53.661707 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:53.661712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:53.661714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:53.661984 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:53.662147 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:53.662892 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 03:11:53.663131 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":273,"threads_started":1}
I20260811 03:11:53.664489 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:53.646279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.3:54551)
0811 03:11:53.647072 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:53.647109 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:53.648859 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:53.649592 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:53.649601 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:53.650104 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:53.650726 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:53.650743 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:53.651951 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:53.651957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:53.653394 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:53.653406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:53.654421 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:53.654431 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:53.654672 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:53.655384 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:53.655419 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:53.657641 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:53.660534 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:53.660543 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:53.660559 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:53.660990 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:53.661378 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:53.661383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:53.661387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:53.661576 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:53.662127 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:53.662135 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:53.662535 (+ 400us) client_negotiation.cc:769] Sending connection context
0811 03:11:53.662797 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 03:11:53.663325 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":292,"threads_started":1}
I20260811 03:11:53.665908 18826 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42751
I20260811 03:11:53.666380 18826 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:53.667353 18826 heartbeater.cc:511] Master 127.11.60.62:42751 requested a full tablet report, sending...
I20260811 03:11:53.669028 18315 ts_manager.cc:194] Registered new tserver with Master: 884a718d1b9b4654a00529bffe0ba706 (127.11.60.3:42001)
I20260811 03:11:53.669900 18315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:54551
I20260811 03:11:53.681442 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.60.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.61.keytab.
Entry for principal oryx/127.11.60.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.61.keytab.
I20260811 03:11:53.743862 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.61:42747
--webserver_interface=127.11.60.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.61.keytab
--principal=oryx/127.11.60.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:42751,127.11.60.61:42747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:54.100157 18834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:54.100487 18834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:54.100584 18834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:54.108435 18834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:54.108621 18834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:54.108672 18834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:54.108724 18834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:54.108769 18834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:54.117274 18673 heartbeater.cc:503] Master 127.11.60.62:42751 was elected leader, sending a full tablet report...
I20260811 03:11:54.120388 18834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:42751,127.11.60.61:42747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.61.keytab
--principal=oryx/127.11.60.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.61:42747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.60.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
W20260811 03:11:54.120777 18834 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.60.62:42751,127.11.60.61:42747'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 03:11:54.121906 18834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:54.123474 18834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:54.136941 18840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:54.136994 18844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:54.136905 18841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:54.138058 18834 server_base.cc:1034] running on GCE node
I20260811 03:11:54.138976 18834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:54.140596 18834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:54.141878 18834 hybrid_clock.cc:648] HybridClock initialized: now 1786417914141787 us; error 66 us; skew 500 ppm
I20260811 03:11:54.142254 18834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:54 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417914, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:54.145694 18834 init.cc:377] Logged in from keytab as oryx/127.11.60.61@KRBTEST.COM (short username oryx)
I20260811 03:11:54.147419 18834 webserver.cc:460] Webserver started at http://127.11.60.61:37791/ using document root <none> and password file <none>
I20260811 03:11:54.148165 18834 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:54.148260 18834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:54.148548 18834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:54.151788 18834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data/instance:
uuid: "6f2b972980a94c26b2d8932f832ed983"
format_stamp: "Formatted at 2026-08-11 03:11:54 on dist-test-slave-tm1g"
server_key: "9a68709b96dc8fd4045d1f1fb9dd19d5"
server_key_iv: "31c8ba5804da1630e883d3b47762fbca"
server_key_version: "encryptionkey@0"
I20260811 03:11:54.152590 18834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal/instance:
uuid: "6f2b972980a94c26b2d8932f832ed983"
format_stamp: "Formatted at 2026-08-11 03:11:54 on dist-test-slave-tm1g"
server_key: "9a68709b96dc8fd4045d1f1fb9dd19d5"
server_key_iv: "31c8ba5804da1630e883d3b47762fbca"
server_key_version: "encryptionkey@0"
I20260811 03:11:54.158429 18834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 03:11:54.162709 18852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:54.164397 18834 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 03:11:54.164573 18834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal
uuid: "6f2b972980a94c26b2d8932f832ed983"
format_stamp: "Formatted at 2026-08-11 03:11:54 on dist-test-slave-tm1g"
server_key: "9a68709b96dc8fd4045d1f1fb9dd19d5"
server_key_iv: "31c8ba5804da1630e883d3b47762fbca"
server_key_version: "encryptionkey@0"
I20260811 03:11:54.164778 18834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:54.184530 18834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:54.189340 18834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:54.189638 18834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:54.210739 18834 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.61:42747
I20260811 03:11:54.210747 18908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.61:42747 every 8 connection(s)
I20260811 03:11:54.212265 18834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
I20260811 03:11:54.216534 18909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:54.221556 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 18834
I20260811 03:11:54.224448 18909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:11:54 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.61@KRBTEST.COM
Aug 11 03:11:54 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417914, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.61@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:11:54.246995 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.229179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.11.60.62:42751)
0811 03:11:54.229692 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:54.229700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:54.230222 (+ 522us) server_negotiation.cc:407] Connection header received
0811 03:11:54.230526 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:54.230537 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:54.230613 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:54.230735 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:54.232119 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.233347 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.235442 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.235852 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.235959 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.240258 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:54.240267 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:54.240275 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:54.240326 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:54.243904 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.244685 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.244691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.244694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.244822 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.245281 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.245286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.245289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.245623 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:54.245786 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:54.246364 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 03:11:54.246602 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":268,"threads_started":1}
I20260811 03:11:54.247938 18914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.229135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:57310)
0811 03:11:54.230083 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:54.230128 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:54.230373 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:54.230806 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:54.230813 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:54.231602 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:54.231933 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.231966 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.233502 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.233508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.235285 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.235304 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.236112 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.236122 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.236353 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.237030 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:54.237067 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:54.239982 (+ 2915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:54.244043 (+ 4061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.244065 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.244079 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.244546 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.244943 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.244949 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.244953 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.245158 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.245765 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:54.245773 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:54.245975 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 03:11:54.246227 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 03:11:54.246846 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":158,"spinlock_wait_cycles":25856,"thread_start_us":558,"threads_started":1}
I20260811 03:11:54.248028 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.224105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40284 (local address 127.11.60.61:42747)
0811 03:11:54.224968 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:54.224983 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:54.225026 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:11:54.226744 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:54.226769 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:54.227239 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:54.227568 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:54.229384 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.231261 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.232943 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.233843 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.233984 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.239172 (+ 5188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:54.239184 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:54.239202 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:54.239257 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:54.243352 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.244188 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.244196 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.244199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.244376 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.244809 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.244814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.244818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.245441 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:54.245763 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:54.246179 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 03:11:54.246686 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":639,"thread_start_us":390,"threads_started":1}
I20260811 03:11:54.250190 18909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:11:54 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417914, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.61@KRBTEST.COM for oryx/127.11.60.61@KRBTEST.COM
I20260811 03:11:54.263366 18914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.250951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42747 (local address 127.0.0.1:40290)
0811 03:11:54.251112 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:54.251128 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:54.251207 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:54.251850 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:54.251854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:54.252247 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:54.252374 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.252382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.253412 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.253417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.254283 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.254293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.255050 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.255056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.255247 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.255812 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:54.255829 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:54.257962 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:54.260968 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.260972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.260974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.261350 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.261745 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.261749 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.261751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.261903 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.262575 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:54.262579 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:54.262706 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 03:11:54.262828 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 03:11:54.263075 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 03:11:54.263486 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.251461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.11.60.61:42747)
0811 03:11:54.251630 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:54.251634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:54.251648 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:11:54.251682 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:54.251685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:54.251733 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:54.251834 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:54.252518 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.253255 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.254473 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.254901 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.255031 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.258144 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:54.258149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:54.258152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:54.258181 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:54.260828 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.261477 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.261482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.261485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.261613 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.262047 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.262051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.262053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.262440 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:54.262558 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:54.262941 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 03:11:54.263210 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":59}
I20260811 03:11:54.268744 18909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Bootstrap starting.
I20260811 03:11:54.272326 18909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:54.273442 18909 log.cc:824] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:54.276562 18909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: No bootstrap required, opened a new log
I20260811 03:11:54.284106 18909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } }
I20260811 03:11:54.284439 18909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:54.284569 18909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2b972980a94c26b2d8932f832ed983, State: Initialized, Role: FOLLOWER
I20260811 03:11:54.285174 18909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } }
I20260811 03:11:54.286074 18916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } } }
I20260811 03:11:54.286504 18916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:11:54.287258 18909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 03:11:54.297019 18929 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:11:54.297168 18929 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:11:54.300876 18834 master_runner.cc:428] Detected that this master 6f2b972980a94c26b2d8932f832ed983 is joining an existing cluster
I20260811 03:11:54.300997 18834 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.60.61:42747
I20260811 03:11:54.326339 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.305788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42747 (local address 127.0.0.1:40304)
0811 03:11:54.307417 (+ 1629us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:54.307454 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:54.307669 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:54.308439 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:54.308446 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:54.309379 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:54.309903 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.309921 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.311844 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.311852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.313842 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.313861 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.315433 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.315442 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.315658 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.316418 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:54.316450 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:54.317866 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:54.322891 (+ 5025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.322896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.322900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.323537 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.324005 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.324010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.324013 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.324247 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.325182 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:54.325186 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:54.325359 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 03:11:54.325522 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 03:11:54.325845 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1423,"thread_start_us":430,"threads_started":1}
I20260811 03:11:54.327283 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.305439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.11.60.61:42747)
0811 03:11:54.305669 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:54.305680 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:54.307554 (+ 1874us) server_negotiation.cc:407] Connection header received
0811 03:11:54.307914 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:54.307922 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:54.308018 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:54.308106 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:54.310101 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.311549 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.313999 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.314947 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.315082 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.318167 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:54.318174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:54.318179 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:54.318220 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:54.322711 (+ 4491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.323718 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.323723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.323727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.323886 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.324370 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.324375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.324379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.324985 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:54.325118 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:54.326676 (+ 1558us) server_negotiation.cc:299] Negotiation successful
0811 03:11:54.326918 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 03:11:54.336799 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.305257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57316 (local address 127.11.60.62:42751)
0811 03:11:54.306427 (+ 1170us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:54.306436 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:54.318500 (+ 12064us) server_negotiation.cc:407] Connection header received
0811 03:11:54.319003 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:54.319011 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:54.319162 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:54.319248 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:54.321315 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.322590 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.325193 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.326043 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.326166 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.329000 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:54.329009 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:54.329017 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:54.329070 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:54.332990 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.333939 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.333947 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.333954 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.334129 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.334675 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.334681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.334686 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.335109 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:54.335255 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:54.336007 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 03:11:54.336312 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986}
I20260811 03:11:54.338069 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.306859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:57316)
0811 03:11:54.318364 (+ 11505us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:54.318395 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:54.318652 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:54.319533 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:54.319539 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:54.320378 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:54.321115 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.321134 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.322803 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.322811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.325003 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.325021 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.326504 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.326512 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.326735 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.327497 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:54.327525 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:54.328772 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:54.333210 (+ 4438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.333215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.333219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.333784 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.334289 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.334297 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.334331 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.334525 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.335380 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:54.335388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:54.335639 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 03:11:54.337104 (+ 1465us) client_negotiation.cc:240] Negotiation successful
0811 03:11:54.337531 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11290,"thread_start_us":9447,"threads_started":1}
WARNING: no policy specified for oryx/127.11.60.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.60@KRBTEST.COM" created.
Aug 11 03:11:54 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417911, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.62@KRBTEST.COM for oryx/127.11.60.61@KRBTEST.COM
I20260811 03:11:54.371438 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.345736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.11.60.61:42747)
0811 03:11:54.345927 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:54.345933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:54.351117 (+ 5184us) server_negotiation.cc:407] Connection header received
0811 03:11:54.351565 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:54.351574 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:54.351718 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:54.351848 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:54.353467 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.354753 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.356820 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.357736 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.357887 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.362756 (+ 4869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:54.362765 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:54.362772 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:54.362827 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:54.367567 (+ 4740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.368712 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.368721 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.368727 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.368936 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.369431 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.369492 (+ 61us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.369495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.369890 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:54.370069 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:54.370834 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 03:11:54.371122 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 03:11:54.371661 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.345539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42747 (local address 127.0.0.1:40308)
0811 03:11:54.350950 (+ 5411us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:54.350998 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:54.351283 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:54.351917 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:54.351926 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:54.352871 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:54.353237 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.353256 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.354915 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.354921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.356624 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.356641 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.357915 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.357921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.358141 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.359011 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:54.359037 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:54.362494 (+ 3457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:54.367759 (+ 5265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.367777 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.367786 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.368562 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.369071 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.369077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.369081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.369274 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.370131 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:54.370141 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:54.370398 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 03:11:54.370654 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 03:11:54.371040 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5119,"thread_start_us":4962,"threads_started":1}
I20260811 03:11:54.374186 18315 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.60.61:42747: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6f2b972980a94c26b2d8932f832ed983"
member_type: NON_VOTER
last_known_addr {
host: "127.11.60.61"
port: 42747
}
attrs {
promote: true
}
}
dest_uuid: "332214273aae4803b3f58ceca1e2b794"
cas_config_opid_index: -1
I20260811 03:11:54.375554 18315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } }
I20260811 03:11:54.379289 18941 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6f2b972980a94c26b2d8932f832ed983 (127.11.60.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } } }
I20260811 03:11:54.380542 18883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.60.keytab.
Entry for principal oryx/127.11.60.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.60.keytab.
I20260811 03:11:54.383785 18941 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.60.61:42747
I20260811 03:11:54.383854 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.60:39749
--webserver_interface=127.11.60.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.60.keytab
--principal=oryx/127.11.60.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:42751,127.11.60.61:42747,127.11.60.60:39749
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:11:54.384466 18883 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Refusing update from remote peer 332214273aae4803b3f58ceca1e2b794: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 03:11:54.384438 18941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } } }
I20260811 03:11:54.384683 18941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:54.385600 18942 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:54.387135 18834 master.cc:537] Master@127.11.60.61:42747 shutting down...
W20260811 03:11:54.389050 18298 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 03:11:54.409534 18298 consensus_peers.cc:487] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 -> Peer 6f2b972980a94c26b2d8932f832ed983 (127.11.60.61:42747): Couldn't send request to peer 6f2b972980a94c26b2d8932f832ed983. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 03:11:54.411456 18834 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:54.411947 18834 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:54.412165 18834 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: stopping tablet replica
I20260811 03:11:54.430166 18834 master.cc:552] Master@127.11.60.61:42747 shutdown complete.
I20260811 03:11:54.430366 18834 master_runner.cc:305] Clearing existing system tablet
I20260811 03:11:54.430874 18834 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:54.430976 18834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:54.434363 18834 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:11:54.436106 18947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:54.436542 18834 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:54.436823 18834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 03:11:54.436923 18834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal
uuid: "6f2b972980a94c26b2d8932f832ed983"
format_stamp: "Formatted at 2026-08-11 03:11:54 on dist-test-slave-tm1g"
server_key: "9a68709b96dc8fd4045d1f1fb9dd19d5"
server_key_iv: "31c8ba5804da1630e883d3b47762fbca"
server_key_version: "encryptionkey@0"
I20260811 03:11:54.442524 18834 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:54.449586 18834 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 03:11:54.449779 18834 log.cc:1196] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 03:11:54.450223 18834 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Deleting consensus metadata
I20260811 03:11:54.450554 18834 master_runner.cc:315] Copying system tablet from 127.11.60.62:42751
I20260811 03:11:54.453158 18834 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: tablet copy: Beginning tablet copy session from remote peer at address 127.11.60.62:42751
I20260811 03:11:54.474148 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.454048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57328 (local address 127.11.60.62:42751)
0811 03:11:54.454242 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:54.454248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:54.458937 (+ 4689us) server_negotiation.cc:407] Connection header received
0811 03:11:54.459139 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:54.459147 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:54.459258 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:54.459382 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:54.460462 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.461608 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.463536 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.464374 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.464541 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.467109 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:54.467117 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:54.467122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:54.467166 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:54.470942 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.471803 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.471809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.471812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.471980 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:54.472441 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:54.472446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:54.472449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:54.472835 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:54.473001 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:54.473536 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 03:11:54.473779 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 03:11:54.474226 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.453992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:57328)
0811 03:11:54.458778 (+ 4786us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:54.458801 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:54.458994 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:54.459426 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:54.459431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:54.459988 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:54.460279 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.460292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.461768 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.461774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.463356 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.463377 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.464600 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.464607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.464787 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.465599 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:54.465627 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:54.466811 (+ 1184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:54.471094 (+ 4283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.471100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.471104 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.471650 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.472107 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:54.472112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:54.472116 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:54.472300 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:54.472978 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:54.472984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:54.473198 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 03:11:54.473374 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 03:11:54.473760 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4625,"thread_start_us":4485,"threads_started":1}
I20260811 03:11:54.475507 18341 tablet_copy_service.cc:140] P 332214273aae4803b3f58ceca1e2b794: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6f2b972980a94c26b2d8932f832ed983 ({username='oryx', principal='oryx/127.11.60.61@KRBTEST.COM'} at 127.0.0.1:57328)
I20260811 03:11:54.475808 18341 tablet_copy_service.cc:161] P 332214273aae4803b3f58ceca1e2b794: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6f2b972980a94c26b2d8932f832ed983 at {username='oryx', principal='oryx/127.11.60.61@KRBTEST.COM'} at 127.0.0.1:57328: session id = 6f2b972980a94c26b2d8932f832ed983-00000000000000000000000000000000
I20260811 03:11:54.480270 18341 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 03:11:54.482722 18834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 03:11:54.490163 18834 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: tablet copy: Starting download of 0 data blocks...
I20260811 03:11:54.490650 18834 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: tablet copy: Starting download of 1 WAL segments...
I20260811 03:11:54.493772 18834 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 03:11:54.498801 18341 tablet_copy_service.cc:342] P 332214273aae4803b3f58ceca1e2b794: Request end of tablet copy session 6f2b972980a94c26b2d8932f832ed983-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.60.61@KRBTEST.COM'} at 127.0.0.1:57328
I20260811 03:11:54.499043 18341 tablet_copy_service.cc:433] P 332214273aae4803b3f58ceca1e2b794: ending tablet copy session 6f2b972980a94c26b2d8932f832ed983-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6f2b972980a94c26b2d8932f832ed983
I20260811 03:11:54.512825 18834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:54.515697 18955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:54.516336 18956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:54.523276 18958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:54.524771 18834 server_base.cc:1034] running on GCE node
I20260811 03:11:54.525115 18834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:54.528846 18834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:54.530103 18834 hybrid_clock.cc:648] HybridClock initialized: now 1786417914530012 us; error 55 us; skew 500 ppm
I20260811 03:11:54.530444 18834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:54 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417914, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:54.533119 18834 init.cc:377] Logged in from keytab as oryx/127.11.60.61@KRBTEST.COM (short username oryx)
I20260811 03:11:54.539943 18834 webserver.cc:460] Webserver started at http://127.11.60.61:36807/ using document root <none> and password file <none>
I20260811 03:11:54.540453 18834 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:54.540570 18834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:54.543880 18834 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:11:54.545989 18966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:54.546518 18834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 03:11:54.546730 18834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal
uuid: "6f2b972980a94c26b2d8932f832ed983"
format_stamp: "Formatted at 2026-08-11 03:11:54 on dist-test-slave-tm1g"
server_key: "9a68709b96dc8fd4045d1f1fb9dd19d5"
server_key_iv: "31c8ba5804da1630e883d3b47762fbca"
server_key_version: "encryptionkey@0"
I20260811 03:11:54.546994 18834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:54.559123 18834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:54.563575 18834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:54.597223 18834 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.61:42747
I20260811 03:11:54.597259 19023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.61:42747 every 8 connection(s)
I20260811 03:11:54.598325 18834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
I20260811 03:11:54.604162 19025 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:11:54.608606 19025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Bootstrap starting.
I20260811 03:11:54.626488 19025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:11:54.627036 19025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Bootstrap complete.
I20260811 03:11:54.627930 19025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } }
I20260811 03:11:54.628185 19025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f2b972980a94c26b2d8932f832ed983, State: Initialized, Role: LEARNER
I20260811 03:11:54.628422 19025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } }
I20260811 03:11:54.629289 19028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } } }
I20260811 03:11:54.629630 19025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:54.629745 19028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: This master's current role is: LEARNER
I20260811 03:11:54.638103 19039 catalog_manager.cc:1259] Loaded cluster ID: 9d125812b9264ea99502cb768c149147
I20260811 03:11:54.638231 19039 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: loading cluster ID for follower catalog manager: success
I20260811 03:11:54.640957 19039 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: acquiring CA information for follower catalog manager: success
I20260811 03:11:54.643584 19039 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:11:54.672580 18826 heartbeater.cc:503] Master 127.11.60.62:42751 was elected leader, sending a full tablet report...
I20260811 03:11:54.838908 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.831471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42747 (local address 127.0.0.1:40310)
0811 03:11:54.831656 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:54.831681 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:54.831825 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:54.832933 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:54.832939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:54.832971 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:54.833224 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.833236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.835303 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.835310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.836606 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:54.836620 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.837985 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.837991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:54.838202 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.838283 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 03:11:54.838451 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 03:11:54.838604 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 03:11:54.838968 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:54.831766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.11.60.61:42747)
0811 03:11:54.832559 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:54.832564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:54.832585 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:11:54.832646 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:54.832649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:54.832719 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:54.832892 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:54.833435 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.835155 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.836772 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:54.837779 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:54.837939 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:54.838206 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:54.838477 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 03:11:54.838620 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":417,"threads_started":1}
I20260811 03:11:54.840265 18996 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 03:11:54.840945 19028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 332214273aae4803b3f58ceca1e2b794. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } } }
I20260811 03:11:54.841228 19028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: This master's current role is: LEARNER
W20260811 03:11:54.898072 18945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:54.898458 18945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:54.898622 18945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:54.906962 18945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:11:54.907140 18945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:54.907265 18945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:54.907364 18945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:11:54.907413 18945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:11:54.918727 18945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:42751,127.11.60.61:42747,127.11.60.60:39749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.60.keytab
--principal=oryx/127.11.60.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.60:39749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.60.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:54.920090 18945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:54.921432 18945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:54.937058 19046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:54.937503 19049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:54.937783 19047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:54.939394 18945 server_base.cc:1034] running on GCE node
I20260811 03:11:54.940317 18945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:54.941933 18945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:54.943333 18945 hybrid_clock.cc:648] HybridClock initialized: now 1786417914943178 us; error 53 us; skew 500 ppm
I20260811 03:11:54.943888 18945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:54 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417914, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:54.948133 18945 init.cc:377] Logged in from keytab as oryx/127.11.60.60@KRBTEST.COM (short username oryx)
I20260811 03:11:54.950003 18945 webserver.cc:460] Webserver started at http://127.11.60.60:45601/ using document root <none> and password file <none>
I20260811 03:11:54.950855 18945 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:54.951071 18945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:54.951494 18945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:11:54.955186 18945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data/instance:
uuid: "80fe5eac26de4645843c11ce986878de"
format_stamp: "Formatted at 2026-08-11 03:11:54 on dist-test-slave-tm1g"
server_key: "3c80fcd2ed614f120e373e32c8f1f297"
server_key_iv: "291743bb91ec9b0bd1f66465609c0e1a"
server_key_version: "encryptionkey@0"
I20260811 03:11:54.956158 18945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal/instance:
uuid: "80fe5eac26de4645843c11ce986878de"
format_stamp: "Formatted at 2026-08-11 03:11:54 on dist-test-slave-tm1g"
server_key: "3c80fcd2ed614f120e373e32c8f1f297"
server_key_iv: "291743bb91ec9b0bd1f66465609c0e1a"
server_key_version: "encryptionkey@0"
I20260811 03:11:54.962361 18945 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 03:11:54.966459 19057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:54.968048 18945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:11:54.968329 18945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal
uuid: "80fe5eac26de4645843c11ce986878de"
format_stamp: "Formatted at 2026-08-11 03:11:54 on dist-test-slave-tm1g"
server_key: "3c80fcd2ed614f120e373e32c8f1f297"
server_key_iv: "291743bb91ec9b0bd1f66465609c0e1a"
server_key_version: "encryptionkey@0"
I20260811 03:11:54.968640 18945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:54.984719 18945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:54.990149 18945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:54.990669 18945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:55.018270 18945 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.60:39749
I20260811 03:11:55.018329 19112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.60:39749 every 8 connection(s)
I20260811 03:11:55.020048 18945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
I20260811 03:11:55.024343 19113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:55.029155 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 18945
I20260811 03:11:55.033474 19113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.60@KRBTEST.COM
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417914, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.60@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:11:55.058817 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.038198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.11.60.62:42751)
0811 03:11:55.039589 (+ 1391us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.039594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.039612 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:11:55.039663 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.039667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.039724 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.039815 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.041590 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.042730 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.045093 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.046362 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.046529 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.051162 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.051168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.051171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.051208 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.055121 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.056122 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.056127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.056129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.056266 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.056692 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.056696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.056698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.057121 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.057272 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.058025 (+ 753us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.058263 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1288,"thread_start_us":1170,"threads_started":1}
I20260811 03:11:55.059504 19118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.037864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:57332)
0811 03:11:55.038765 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.038818 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.039173 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.039933 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.039941 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.040883 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.041429 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.041440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.042913 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.042917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.044884 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.044910 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.046655 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.046661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.046812 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.047568 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.047601 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.050947 (+ 3346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.055266 (+ 4319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.055275 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.055281 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.055993 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.056405 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.056409 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.056412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.056564 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.057338 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.057347 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.057553 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.057780 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.058239 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":151,"thread_start_us":437,"threads_started":1}
I20260811 03:11:55.059523 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.032001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50516 (local address 127.11.60.60:39749)
0811 03:11:55.033164 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.033181 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.033242 (+ 61us) server_negotiation.cc:407] Connection header received
0811 03:11:55.035343 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.035367 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.035811 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.036314 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.038703 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.041018 (+ 2315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.042899 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.044002 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.044206 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.050373 (+ 6167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.050387 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.050404 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.050445 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.054034 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.055246 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.055254 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.055264 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.055572 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.055979 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.055985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.055988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.056729 (+ 741us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.056979 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.057609 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.058113 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":759,"spinlock_wait_cycles":21248,"thread_start_us":493,"threads_started":1}
I20260811 03:11:55.061667 19113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417914, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.60@KRBTEST.COM for oryx/127.11.60.61@KRBTEST.COM
I20260811 03:11:55.076262 19118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.062525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42747 (local address 127.0.0.1:40320)
0811 03:11:55.062725 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.062745 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.062857 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.063257 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.063260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.063570 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.063706 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.063713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.064751 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.064754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.065868 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.065880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.066784 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.066789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.066963 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.067501 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.067517 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.069793 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.073943 (+ 4150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.073947 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.073949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.074413 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.074762 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.074765 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.074767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.074910 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.075567 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.075570 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.075689 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.075802 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.076042 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 03:11:55.076536 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.062609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40320 (local address 127.11.60.61:42747)
0811 03:11:55.062808 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.062812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.062830 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:11:55.063026 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.063032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.063131 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.063221 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.063835 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.064636 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.066021 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.066563 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.066742 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.070005 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.070010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.070014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.070050 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.073792 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.074520 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.074525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.074527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.074663 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.075029 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.075033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.075036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.075453 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.075641 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.075924 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.076179 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 03:11:55.077615 19113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417914, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.60@KRBTEST.COM for oryx/127.11.60.60@KRBTEST.COM
I20260811 03:11:55.091516 19118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.078362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:39749 (local address 127.0.0.1:50526)
0811 03:11:55.078639 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.078659 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.078772 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.079053 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.079056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.079557 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.079746 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.079755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.080912 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.080916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.082130 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.082144 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.082881 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.082885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.083068 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.083717 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.083735 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.086059 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.089161 (+ 3102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.089165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.089168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.089552 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.089880 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.089883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.089886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.090051 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.090614 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.090618 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.090746 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.090903 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.091143 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 03:11:55.091548 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.078524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.11.60.60:39749)
0811 03:11:55.078668 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.078672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.078727 (+ 55us) server_negotiation.cc:407] Connection header received
0811 03:11:55.078867 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.078871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.078925 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.079002 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.079880 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.080776 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.082261 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.082715 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.082848 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.086249 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.086254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.086257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.086285 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.089031 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.089653 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.089656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.089659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.089782 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.090154 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.090157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.090159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.090503 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.090614 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.090996 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.091278 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":51}
I20260811 03:11:55.097307 19113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: Bootstrap starting.
I20260811 03:11:55.100883 19113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:55.102187 19113 log.cc:824] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:55.105234 19113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: No bootstrap required, opened a new log
I20260811 03:11:55.114986 19113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } }
I20260811 03:11:55.115387 19113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:55.115577 19113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80fe5eac26de4645843c11ce986878de, State: Initialized, Role: FOLLOWER
I20260811 03:11:55.116207 19113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } }
I20260811 03:11:55.117374 19121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } } }
I20260811 03:11:55.117826 19121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:11:55.118736 19113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: configured and running, proceeding with master startup.
W20260811 03:11:55.130867 19133 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:11:55.131054 19133 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:11:55.135684 18945 master_runner.cc:428] Detected that this master 80fe5eac26de4645843c11ce986878de is joining an existing cluster
I20260811 03:11:55.135823 18945 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.60.60:39749
I20260811 03:11:55.166855 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 18384
I20260811 03:11:55.167938 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.141351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.11.60.61:42747)
0811 03:11:55.141555 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.141563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.146850 (+ 5287us) server_negotiation.cc:407] Connection header received
0811 03:11:55.147148 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.147156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.147250 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.147326 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.148975 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.150115 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.153191 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.154167 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.154366 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.157202 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.157211 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.157218 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.157264 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.161274 (+ 4010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.162243 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.162249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.162253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.162460 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.162942 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.162949 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.162954 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.163312 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.163474 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.164408 (+ 934us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.164731 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 03:11:55.169292 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.142023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42747 (local address 127.0.0.1:40336)
0811 03:11:55.146727 (+ 4704us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.146755 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.146963 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.147575 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.147579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.148403 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.148781 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.148795 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.150385 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.150390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.152991 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.153008 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.154441 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.154448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.154629 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.155526 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.155555 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.156964 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.161472 (+ 4508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.161477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.161481 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.162092 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.162620 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.162624 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.162628 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.162811 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.163595 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.163609 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.164011 (+ 402us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.168209 (+ 4198us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.168608 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4555,"thread_start_us":4329,"threads_started":1}
I20260811 03:11:55.176056 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.147208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.11.60.60:39749)
0811 03:11:55.147382 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.147387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.157076 (+ 9689us) server_negotiation.cc:407] Connection header received
0811 03:11:55.157274 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.157278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.157341 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.157418 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.159299 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.160500 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.162845 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.163855 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.164003 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.166434 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.166440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.166445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.166477 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.172038 (+ 5561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.173147 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.173152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.173155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.173430 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.174027 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.174033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.174036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.174692 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.174847 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.175412 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.175667 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62,"spinlock_wait_cycles":768}
I20260811 03:11:55.176559 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.141379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57340 (local address 127.11.60.62:42751)
0811 03:11:55.141610 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.141617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.145296 (+ 3679us) server_negotiation.cc:407] Connection header received
0811 03:11:55.145544 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.145554 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.145707 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.145851 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.148490 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.150045 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.158217 (+ 8172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.159031 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.159214 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.162418 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.162426 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.162431 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.162472 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.166788 (+ 4316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.173251 (+ 6463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.173258 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.173261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.173470 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.174040 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.174047 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.174050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.174520 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.174712 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.175496 (+ 784us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.175794 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 03:11:55.177042 19142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.147167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:39749 (local address 127.0.0.1:50540)
0811 03:11:55.156950 (+ 9783us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.156973 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.157165 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.157602 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.157606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.158572 (+ 966us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.159104 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.159120 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.160755 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.160759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.162707 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.162720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.164343 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.164351 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.164522 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.165064 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.165090 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.166203 (+ 1113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.172180 (+ 5977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.172187 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.172192 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.173021 (+ 829us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.173606 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.173610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.173613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.173854 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.174926 (+ 1072us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.174931 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.175134 (+ 203us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.176255 (+ 1121us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.176563 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9550,"thread_start_us":9352,"threads_started":1}
I20260811 03:11:55.178169 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.142015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:57340)
0811 03:11:55.144971 (+ 2956us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.144993 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.145374 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.146118 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.146127 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.147293 (+ 1166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.148148 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.148169 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.150458 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.150467 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.157706 (+ 7239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.157730 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.159783 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.159790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.160002 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.160691 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.160717 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.162149 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.167471 (+ 5322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.167476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.167481 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.173086 (+ 5605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.173635 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.173640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.173643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.173894 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.174831 (+ 937us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.174839 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.175119 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.177033 (+ 1914us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.177605 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2788,"mutex_wait_us":3,"spinlock_wait_cycles":44416,"thread_start_us":2569,"threads_started":1}
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.1@KRBTEST.COM
I20260811 03:11:55.196538 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.168018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36856 (local address 127.11.60.1:39959)
0811 03:11:55.172692 (+ 4674us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.172702 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.172736 (+ 34us) server_negotiation.cc:407] Connection header received
0811 03:11:55.172811 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.172836 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.173181 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.175621 (+ 2440us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.175901 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.178181 (+ 2280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.180026 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.180844 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.181007 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.188111 (+ 7104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.188133 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.188153 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.188208 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.192563 (+ 4355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.193470 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.193485 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.193490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.193670 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.194258 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.194264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.194268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.194860 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.195197 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.195684 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.196062 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4464,"thread_start_us":4201,"threads_started":1}
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417911, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.62@KRBTEST.COM for oryx/127.11.60.60@KRBTEST.COM
I20260811 03:11:55.198268 18516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:55.207369 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.185558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:39749 (local address 127.0.0.1:50556)
0811 03:11:55.185996 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.186016 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.186124 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.187203 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.187211 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.187666 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.187864 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.187900 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.189692 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.189700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.191378 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.191396 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.192445 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.192453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.192641 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.193292 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.193312 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.197856 (+ 4544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.204223 (+ 6367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.204230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.204234 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.204742 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.205189 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.205195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.205199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.205387 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.206081 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.206086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.206272 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.206522 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.206912 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273}
I20260811 03:11:55.207389 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.186610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50556 (local address 127.11.60.60:39749)
0811 03:11:55.186831 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.186838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.186857 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:11:55.186967 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.186971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.187044 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.187175 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.188130 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.189496 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.191548 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.192238 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.192374 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.198083 (+ 5709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.198090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.198095 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.198137 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.204049 (+ 5912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.204900 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.204906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.204911 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.205060 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.205624 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.205628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.205630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.205939 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.206119 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.206651 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.206963 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 03:11:55.209589 18315 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.60.60:39749: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "80fe5eac26de4645843c11ce986878de"
member_type: NON_VOTER
last_known_addr {
host: "127.11.60.60"
port: 39749
}
attrs {
promote: true
}
}
dest_uuid: "332214273aae4803b3f58ceca1e2b794"
cas_config_opid_index: 5
I20260811 03:11:55.210755 18315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } }
I20260811 03:11:55.212787 18941 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 80fe5eac26de4645843c11ce986878de (127.11.60.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.219594 18996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 LEARNER]: Refusing update from remote peer 332214273aae4803b3f58ceca1e2b794: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 03:11:55.220481 19087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:55.226874 19087 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 FOLLOWER]: Refusing update from remote peer 332214273aae4803b3f58ceca1e2b794: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 03:11:55.240895 19147 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 03:11:55.242215 18941 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.60.60:39749
I20260811 03:11:55.243224 18945 master.cc:537] Master@127.11.60.60:39749 shutting down...
W20260811 03:11:55.243148 19065 tablet.cc:2369] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:55.245512 19147 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:55.251708 19145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.252192 19145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:55.258484 18945 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:55.259045 18945 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:55.259313 18945 tablet_replica.cc:330] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: stopping tablet replica
I20260811 03:11:55.268345 18945 master.cc:552] Master@127.11.60.60:39749 shutdown complete.
I20260811 03:11:55.268473 18945 master_runner.cc:305] Clearing existing system tablet
I20260811 03:11:55.268090 18996 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 80fe5eac26de4645843c11ce986878de (127.11.60.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.268939 18945 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:55.269070 18945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:55.271768 18945 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 03:11:55.272881 19028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: NON_VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: true } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.273177 19028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: This master's current role is: LEARNER
I20260811 03:11:55.274206 19152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:55.274763 18945 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:55.275085 18945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:11:55.275215 18945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal
uuid: "80fe5eac26de4645843c11ce986878de"
format_stamp: "Formatted at 2026-08-11 03:11:54 on dist-test-slave-tm1g"
server_key: "3c80fcd2ed614f120e373e32c8f1f297"
server_key_iv: "291743bb91ec9b0bd1f66465609c0e1a"
server_key_version: "encryptionkey@0"
I20260811 03:11:55.279397 19145 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794: attempting to promote NON_VOTER 6f2b972980a94c26b2d8932f832ed983 to VOTER
I20260811 03:11:55.280113 18945 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:55.280403 19145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } }
I20260811 03:11:55.283202 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.282479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:39749 (local address 127.0.0.1:50558)
0811 03:11:55.282882 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.283026 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239}
I20260811 03:11:55.283618 18996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 LEARNER]: Refusing update from remote peer 332214273aae4803b3f58ceca1e2b794: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 03:11:55.284875 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.284213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:39749 (local address 127.0.0.1:50570)
0811 03:11:55.284416 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.284514 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 03:11:55.284574 19146 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 03:11:55.285099 18298 consensus_peers.cc:487] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 -> Peer 80fe5eac26de4645843c11ce986878de (127.11.60.60:39749): Couldn't send request to peer 80fe5eac26de4645843c11ce986878de. Status: Network error: Client connection negotiation failed: client connection to 127.11.60.60:39749: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 03:11:55.287995 19146 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6f2b972980a94c26b2d8932f832ed983 (127.11.60.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.291185 18945 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 03:11:55.291450 18945 log.cc:1196] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 03:11:55.292116 18945 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: Deleting consensus metadata
I20260811 03:11:55.292526 18945 master_runner.cc:315] Copying system tablet from 127.11.60.62:42751
I20260811 03:11:55.293154 18996 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6f2b972980a94c26b2d8932f832ed983 (127.11.60.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.299422 19147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.299866 19147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:55.300307 19147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.300649 19147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:55.301469 19028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.301851 19028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:11:55.305689 18945 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: tablet copy: Beginning tablet copy session from remote peer at address 127.11.60.62:42751
I20260811 03:11:55.321916 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.306629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57344 (local address 127.11.60.62:42751)
0811 03:11:55.306901 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.306905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.307565 (+ 660us) server_negotiation.cc:407] Connection header received
0811 03:11:55.307846 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.307852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.307932 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.308051 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.309186 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.310131 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.311616 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.312276 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.312431 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.315019 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.315025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.315029 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.315065 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.318559 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.319409 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.319416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.319422 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.319646 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.320132 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.320138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.320142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.320524 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.320704 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.321169 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.321440 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260811 03:11:55.322374 19159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.306632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:57344)
0811 03:11:55.307444 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.307471 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.307673 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.308085 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.308090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.308790 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.309027 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.309038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.310379 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.310385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.311444 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.311457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.312470 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.312477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.312859 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.313701 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.313737 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.314770 (+ 1033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.318706 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.318713 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.318718 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.319256 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.319776 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.319781 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.319786 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.319962 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.320665 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.320670 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.320870 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.321044 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.321440 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":546,"threads_started":1}
I20260811 03:11:55.323107 18341 tablet_copy_service.cc:140] P 332214273aae4803b3f58ceca1e2b794: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 80fe5eac26de4645843c11ce986878de ({username='oryx', principal='oryx/127.11.60.60@KRBTEST.COM'} at 127.0.0.1:57344)
I20260811 03:11:55.323345 18341 tablet_copy_service.cc:161] P 332214273aae4803b3f58ceca1e2b794: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 80fe5eac26de4645843c11ce986878de at {username='oryx', principal='oryx/127.11.60.60@KRBTEST.COM'} at 127.0.0.1:57344: session id = 80fe5eac26de4645843c11ce986878de-00000000000000000000000000000000
I20260811 03:11:55.326231 18341 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 03:11:55.328078 18945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 03:11:55.335767 18945 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: tablet copy: Starting download of 0 data blocks...
I20260811 03:11:55.336164 18945 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: tablet copy: Starting download of 1 WAL segments...
I20260811 03:11:55.339352 18945 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 03:11:55.343413 18341 tablet_copy_service.cc:342] P 332214273aae4803b3f58ceca1e2b794: Request end of tablet copy session 80fe5eac26de4645843c11ce986878de-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.60.60@KRBTEST.COM'} at 127.0.0.1:57344
I20260811 03:11:55.343585 18341 tablet_copy_service.cc:433] P 332214273aae4803b3f58ceca1e2b794: ending tablet copy session 80fe5eac26de4645843c11ce986878de-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 80fe5eac26de4645843c11ce986878de
I20260811 03:11:55.349128 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 18384
I20260811 03:11:55.366782 18945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:55.369972 19160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:55.376019 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:39959
--local_ip_for_outbound_sockets=127.11.60.1
--tserver_master_addrs=127.11.60.62:42751
--webserver_port=40515
--webserver_interface=127.11.60.1
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab
--principal=oryx/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.60.62:42751,127.11.60.61:42747
--tserver_master_addrs=127.11.60.62:42751,127.11.60.61:42747,127.11.60.60:39749 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:11:55.376361 19161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:55.390528 19163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:55.391361 18945 server_base.cc:1034] running on GCE node
I20260811 03:11:55.391753 18945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:55.392525 18945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:55.393767 18945 hybrid_clock.cc:648] HybridClock initialized: now 1786417915393691 us; error 56 us; skew 500 ppm
I20260811 03:11:55.394078 18945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417915, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:55.396636 18945 init.cc:377] Logged in from keytab as oryx/127.11.60.60@KRBTEST.COM (short username oryx)
I20260811 03:11:55.399588 18945 webserver.cc:460] Webserver started at http://127.11.60.60:40297/ using document root <none> and password file <none>
I20260811 03:11:55.400009 18945 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:55.400156 18945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:55.404052 18945 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260811 03:11:55.406447 19173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:55.406932 18945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260811 03:11:55.407130 18945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal
uuid: "80fe5eac26de4645843c11ce986878de"
format_stamp: "Formatted at 2026-08-11 03:11:54 on dist-test-slave-tm1g"
server_key: "3c80fcd2ed614f120e373e32c8f1f297"
server_key_iv: "291743bb91ec9b0bd1f66465609c0e1a"
server_key_version: "encryptionkey@0"
I20260811 03:11:55.407606 18945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:55.422546 18945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:55.427104 18945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:55.460950 18945 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.60:39749
I20260811 03:11:55.460989 19225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.60:39749 every 8 connection(s)
I20260811 03:11:55.462528 18945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
I20260811 03:11:55.468009 19226 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:11:55.471652 19226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: Bootstrap starting.
I20260811 03:11:55.491271 19226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:11:55.491923 19226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: Bootstrap complete.
I20260811 03:11:55.493180 19226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } }
I20260811 03:11:55.493523 19226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 80fe5eac26de4645843c11ce986878de, State: Initialized, Role: LEARNER
I20260811 03:11:55.493803 19226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } }
I20260811 03:11:55.494740 19229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.495111 19226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:11:55.495251 19229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: This master's current role is: LEARNER
I20260811 03:11:55.505636 19242 catalog_manager.cc:1259] Loaded cluster ID: 9d125812b9264ea99502cb768c149147
I20260811 03:11:55.505766 19242 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: loading cluster ID for follower catalog manager: success
I20260811 03:11:55.509473 19242 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: acquiring CA information for follower catalog manager: success
I20260811 03:11:55.512112 19242 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:11:55.795460 19243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.788128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:39749 (local address 127.0.0.1:50584)
0811 03:11:55.788702 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.788730 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.788900 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.789508 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.789512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.789539 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:55.789754 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.789763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.791633 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.791637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.793050 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.793067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.794425 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.794432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.794641 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.794690 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.794839 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.795031 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":287,"threads_started":1}
I20260811 03:11:55.795507 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.788288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50584 (local address 127.11.60.60:39749)
0811 03:11:55.789091 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.789098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.789120 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:11:55.789198 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.789203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.789301 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.789439 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:55.789953 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.791488 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.793250 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.794194 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.794368 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.794543 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.794935 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.795106 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":428,"threads_started":1}
I20260811 03:11:55.796494 19200 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 03:11:55.797137 19229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 332214273aae4803b3f58ceca1e2b794. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:55.797482 19229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: This master's current role is: LEARNER
W20260811 03:11:55.802721 19165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:55.802997 19165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:55.803097 19165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:55.811036 19165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:55.811154 19165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:55.811276 19165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:11:55.821835 19165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab
--principal=oryx/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:39959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=40515
--tserver_master_addrs=127.11.60.62:42751,127.11.60.61:42747,127.11.60.60:39749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:55.823282 19165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:55.824793 19165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:55.839161 19249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:55.839171 19250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:55.839327 19252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:55.840094 19165 server_base.cc:1034] running on GCE node
I20260811 03:11:55.840749 19165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:55.841729 19165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:55.842963 19165 hybrid_clock.cc:648] HybridClock initialized: now 1786417915842896 us; error 62 us; skew 500 ppm
I20260811 03:11:55.843290 19165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417915, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:55.846936 19165 init.cc:377] Logged in from keytab as oryx/127.11.60.1@KRBTEST.COM (short username oryx)
I20260811 03:11:55.848723 19165 webserver.cc:460] Webserver started at http://127.11.60.1:40515/ using document root <none> and password file <none>
I20260811 03:11:55.849453 19165 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:55.849629 19165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:55.855511 19165 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 03:11:55.858928 19259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:55.860634 19165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:11:55.860795 19165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "9cfcb8537b49453096cea3274246b3d2"
format_stamp: "Formatted at 2026-08-11 03:11:52 on dist-test-slave-tm1g"
server_key: "00ed5f9546a70176c291f0d09349361d"
server_key_iv: "49169756f7f810e19816099ecf12abf8"
server_key_version: "encryptionkey@0"
I20260811 03:11:55.862123 19165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:55.884788 19165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:55.889595 19165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:55.889884 19165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:55.890998 19165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:55.892529 19165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:55.892624 19165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:55.892722 19165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:55.892786 19165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:55.933112 19165 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:39959
I20260811 03:11:55.933211 19379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:39959 every 8 connection(s)
I20260811 03:11:55.934674 19165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:11:55.944597 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 19165
I20260811 03:11:55.945034 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 18527
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417915, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.60@KRBTEST.COM
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417915, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417915, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.61@KRBTEST.COM
Aug 11 03:11:55 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.2@KRBTEST.COM
I20260811 03:11:55.963948 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.937956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:37083 (local address 127.11.60.60:39749)
0811 03:11:55.938183 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.938190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.939504 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 03:11:55.941506 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.941514 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.941611 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.941710 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.944541 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.946141 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.949222 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.949909 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.950008 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.955834 (+ 5826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.955840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.955845 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.955889 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.959868 (+ 3979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.960874 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.960880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.960884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.961052 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.961651 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.961657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.961660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.962013 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.962155 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.963338 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.963583 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 03:11:55.963967 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.937653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:60243 (local address 127.11.60.62:42751)
0811 03:11:55.938439 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.938444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.942448 (+ 4004us) server_negotiation.cc:407] Connection header received
0811 03:11:55.942648 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.942655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.942741 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.942821 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.944522 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.945751 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.949902 (+ 4151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.950584 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.950682 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.955850 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.955857 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.955862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.955903 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.959692 (+ 3789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.960874 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.960880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.960884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.961052 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.961651 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.961657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.961660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.962022 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.962165 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.963338 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.963580 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":494,"threads_started":1}
I20260811 03:11:55.966763 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.938407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:39749 (local address 127.11.60.1:37083)
0811 03:11:55.939331 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.939375 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.941201 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.941978 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.941993 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.942938 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.943916 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.943947 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.946471 (+ 2524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.946479 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.949052 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.949068 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.950363 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.950371 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.950674 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.951482 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.951545 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.955553 (+ 4008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.960019 (+ 4466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.960026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.960032 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.960724 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.961207 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.961212 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.961216 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.961517 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.962265 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.962274 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.962859 (+ 585us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.964258 (+ 1399us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.965347 (+ 1089us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":451,"spinlock_wait_cycles":12928,"thread_start_us":3460,"threads_started":1}
I20260811 03:11:55.966768 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.938514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.1:60243)
0811 03:11:55.942328 (+ 3814us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.942348 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.942487 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.943039 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.943047 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.943663 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.944334 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.944347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.947943 (+ 3596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.947948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.949715 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.949731 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.950985 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.950994 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.951152 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.951720 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.951739 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.955553 (+ 3814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.959878 (+ 4325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.959887 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.959909 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.960724 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.961207 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.961212 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.961216 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.961512 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.962265 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.962274 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.962859 (+ 585us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.964258 (+ 1399us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.965058 (+ 800us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3643,"mutex_wait_us":499,"spinlock_wait_cycles":60928,"thread_start_us":429,"threads_started":1}
I20260811 03:11:55.967841 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.942658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42747 (local address 127.11.60.1:60601)
0811 03:11:55.944214 (+ 1556us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:55.944236 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:55.944349 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:55.948490 (+ 4141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:55.948497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:55.949292 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:55.949725 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.949740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.951856 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.951861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.953563 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:55.953580 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.955476 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.955483 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:55.955713 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.956333 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:55.956355 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:55.959272 (+ 2917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:55.963681 (+ 4409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.963689 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.963695 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.964705 (+ 1010us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.965210 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:55.965215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:55.965219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:55.965416 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:55.966147 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:55.966151 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:55.966375 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 03:11:55.966532 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 03:11:55.967375 (+ 843us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1427,"thread_start_us":1280,"threads_started":1}
I20260811 03:11:55.968863 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.942669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:60601 (local address 127.11.60.61:42747)
0811 03:11:55.947979 (+ 5310us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.947986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.948011 (+ 25us) server_negotiation.cc:407] Connection header received
0811 03:11:55.948070 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.948075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.948170 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.948281 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.949927 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.951670 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.953738 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.954537 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.954678 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.959563 (+ 4885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.959571 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.959577 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.959629 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.963509 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.964865 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.964873 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.964878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.965037 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.965565 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.965570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.965573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.965938 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.966073 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.968264 (+ 2191us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.968513 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5189,"thread_start_us":5085,"threads_started":1}
I20260811 03:11:55.968587 19381 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42751
I20260811 03:11:55.969398 19381 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:55.970377 19381 heartbeater.cc:511] Master 127.11.60.62:42751 requested a full tablet report, sending...
I20260811 03:11:55.970458 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:55.946977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48778 (local address 127.11.60.2:43797)
0811 03:11:55.947646 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:55.947655 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:55.947689 (+ 34us) server_negotiation.cc:407] Connection header received
0811 03:11:55.947770 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:55.947779 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:55.948101 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:55.950529 (+ 2428us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:55.950794 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.952346 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.954157 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:55.955043 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:55.955177 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:55.962904 (+ 7727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:55.962923 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:55.962942 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:55.962991 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:55.967075 (+ 4084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.968041 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.968049 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.968052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.968215 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:55.968697 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:55.968704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:55.968709 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:55.969154 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:55.969505 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:55.969716 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 03:11:55.969985 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":398,"threads_started":1}
I20260811 03:11:55.968712 19380 heartbeater.cc:347] Connected to a master server at 127.11.60.60:39749
I20260811 03:11:55.971233 19380 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:55.971580 18668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:55.971668 19380 heartbeater.cc:511] Master 127.11.60.60:39749 requested a full tablet report, sending...
I20260811 03:11:55.972376 18315 ts_manager.cc:194] Re-registered known tserver with Master: 9cfcb8537b49453096cea3274246b3d2 (127.11.60.1:39959)
I20260811 03:11:55.973516 19190 ts_manager.cc:194] Registered new tserver with Master: 9cfcb8537b49453096cea3274246b3d2 (127.11.60.1:39959)
I20260811 03:11:55.973913 18315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:60243
I20260811 03:11:55.975222 19382 heartbeater.cc:347] Connected to a master server at 127.11.60.61:42747
I20260811 03:11:55.975361 19382 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:55.976063 19382 heartbeater.cc:511] Master 127.11.60.61:42747 requested a full tablet report, sending...
I20260811 03:11:55.978147 18985 ts_manager.cc:194] Registered new tserver with Master: 9cfcb8537b49453096cea3274246b3d2 (127.11.60.1:39959)
I20260811 03:11:56.061278 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 18527
I20260811 03:11:56.082017 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:43797
--local_ip_for_outbound_sockets=127.11.60.2
--tserver_master_addrs=127.11.60.62:42751
--webserver_port=36309
--webserver_interface=127.11.60.2
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab
--principal=oryx/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.60.62:42751,127.11.60.61:42747
--tserver_master_addrs=127.11.60.62:42751,127.11.60.61:42747,127.11.60.60:39749 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:11:56.189179 19146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: NON_VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: true } } }
I20260811 03:11:56.189711 19146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:56.191334 19398 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794: attempting to promote NON_VOTER 80fe5eac26de4645843c11ce986878de to VOTER
I20260811 03:11:56.192139 19398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } }
I20260811 03:11:56.194483 19200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 LEARNER]: Refusing update from remote peer 332214273aae4803b3f58ceca1e2b794: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 03:11:56.194553 18996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Refusing update from remote peer 332214273aae4803b3f58ceca1e2b794: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 03:11:56.195092 19146 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 03:11:56.195576 19396 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 03:11:56.199077 19398 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 80fe5eac26de4645843c11ce986878de (127.11.60.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:11:56.199190 18996 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 80fe5eac26de4645843c11ce986878de (127.11.60.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:11:56.201092 19200 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 80fe5eac26de4645843c11ce986878de (127.11.60.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:11:56.203073 19398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:11:56.203383 19398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:56.203639 19398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:11:56.203967 19398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:56.204434 19398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:11:56.204941 19398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794 [sys.catalog]: This master's current role is: LEADER
I20260811 03:11:56.207525 19229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:11:56.207836 19229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:11:56.208840 19401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "332214273aae4803b3f58ceca1e2b794" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:11:56.209256 19401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 03:11:56.442155 19395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:56.442469 19395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:56.442569 19395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:56.450145 19395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:56.450371 19395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:56.450572 19395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:11:56.461464 19395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab
--principal=oryx/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:43797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=36309
--tserver_master_addrs=127.11.60.62:42751,127.11.60.61:42747,127.11.60.60:39749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:56.462909 19395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:56.464471 19395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:56.479471 19407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:56.479526 19406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:56.480202 19409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:56.480628 19395 server_base.cc:1034] running on GCE node
I20260811 03:11:56.481268 19395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:56.482352 19395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:56.483587 19395 hybrid_clock.cc:648] HybridClock initialized: now 1786417916483530 us; error 52 us; skew 500 ppm
I20260811 03:11:56.483875 19395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:56 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417916, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:56.487272 19395 init.cc:377] Logged in from keytab as oryx/127.11.60.2@KRBTEST.COM (short username oryx)
I20260811 03:11:56.488945 19395 webserver.cc:460] Webserver started at http://127.11.60.2:36309/ using document root <none> and password file <none>
I20260811 03:11:56.489692 19395 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:56.489862 19395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:56.496022 19395 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 03:11:56.499185 19417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:56.500692 19395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:11:56.500883 19395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "ee8e84046be747fc98b50003173b3342"
format_stamp: "Formatted at 2026-08-11 03:11:52 on dist-test-slave-tm1g"
server_key: "87ad67f7816d7901e7082e383467fe42"
server_key_iv: "b775c7fab26fa860602fdc60beeb4d59"
server_key_version: "encryptionkey@0"
I20260811 03:11:56.502030 19395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:56.530292 19395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:56.535317 19395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:56.535640 19395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:56.536720 19395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:56.538287 19395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:56.538434 19395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:56.538580 19395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:56.538659 19395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:56.573360 19395 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:43797
I20260811 03:11:56.573464 19545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:43797 every 8 connection(s)
I20260811 03:11:56.574911 19395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:11:56.583379 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 19395
I20260811 03:11:56.583778 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 18681
Aug 11 03:11:56 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417916, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
Aug 11 03:11:56 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417916, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.61@KRBTEST.COM
I20260811 03:11:56.617758 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:56.578164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:36979 (local address 127.11.60.62:42751)
0811 03:11:56.578707 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:56.578717 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:56.596742 (+ 18025us) server_negotiation.cc:407] Connection header received
0811 03:11:56.597037 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:56.597044 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:56.597126 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:56.597202 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:56.599903 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.601105 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.603749 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.604421 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.604514 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:56.610374 (+ 5860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:56.610382 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:56.610387 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:56.610433 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:56.613820 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:56.614839 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:56.614844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:56.614847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:56.615061 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:56.615646 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:56.615655 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:56.615659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:56.616036 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:56.616192 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:56.617178 (+ 986us) server_negotiation.cc:299] Negotiation successful
0811 03:11:56.617472 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":252,"threads_started":1}
I20260811 03:11:56.619298 19554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:56.579810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.2:36979)
0811 03:11:56.596617 (+ 16807us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:56.596639 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:56.596865 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:56.597708 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:56.597722 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:56.598753 (+ 1031us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:56.599654 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:56.599680 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.601335 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.601341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:56.603578 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:56.603594 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.604893 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.604900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:56.605211 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:56.606117 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:56.606149 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:56.610053 (+ 3904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:56.613958 (+ 3905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:56.613978 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:56.613996 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:56.614701 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:56.615190 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:56.615196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:56.615200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:56.615486 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:56.616171 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:56.616179 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:56.616678 (+ 499us) client_negotiation.cc:769] Sending connection context
0811 03:11:56.617041 (+ 363us) client_negotiation.cc:240] Negotiation successful
0811 03:11:56.617694 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16672,"thread_start_us":16220,"threads_started":1}
I20260811 03:11:56.620944 19547 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42751
I20260811 03:11:56.621304 19547 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:56.622342 19547 heartbeater.cc:511] Master 127.11.60.62:42751 requested a full tablet report, sending...
I20260811 03:11:56.624490 18315 ts_manager.cc:194] Re-registered known tserver with Master: ee8e84046be747fc98b50003173b3342 (127.11.60.2:43797)
I20260811 03:11:56.625761 19552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:56.578994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42747 (local address 127.11.60.2:58975)
0811 03:11:56.585799 (+ 6805us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:56.585884 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:56.589224 (+ 3340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:56.607007 (+ 17783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:56.607018 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:56.607574 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:56.607895 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:56.607913 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.609954 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.609960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:56.611525 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:56.611540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.614377 (+ 2837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.614384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:56.614617 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:56.615292 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:56.615313 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:56.619168 (+ 3855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:56.622765 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:56.622770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:56.622774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:56.623136 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:56.623587 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:56.623595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:56.623598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:56.623854 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:56.624585 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:56.624591 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:56.624742 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 03:11:56.624882 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 03:11:56.625211 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6350,"mutex_wait_us":42,"spinlock_wait_cycles":71424,"thread_start_us":6065,"threads_started":1}
I20260811 03:11:56.625888 18315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:36979
Aug 11 03:11:56 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.3@KRBTEST.COM
I20260811 03:11:56.626680 19551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:56.578189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:58975 (local address 127.11.60.61:42747)
0811 03:11:56.606397 (+ 28208us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:56.606404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:56.606424 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:11:56.606490 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:56.606495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:56.606762 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:56.608064 (+ 1302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:56.608307 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.609763 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.613498 (+ 3735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.614140 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.614277 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:56.619445 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:56.619452 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:56.619457 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:56.619509 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:56.622627 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:56.623263 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:56.623290 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:56.623296 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:56.623450 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:56.624030 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:56.624035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:56.624038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:56.624439 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:56.626033 (+ 1594us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:56.626142 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 03:11:56.626413 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28033,"thread_start_us":28344,"threads_started":1}
Aug 11 03:11:56 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417916, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.60@KRBTEST.COM
I20260811 03:11:56.627620 19548 heartbeater.cc:347] Connected to a master server at 127.11.60.61:42747
I20260811 03:11:56.627791 19548 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:56.628216 19548 heartbeater.cc:511] Master 127.11.60.61:42747 requested a full tablet report, sending...
I20260811 03:11:56.629544 18985 ts_manager.cc:194] Registered new tserver with Master: ee8e84046be747fc98b50003173b3342 (127.11.60.2:43797)
I20260811 03:11:56.636023 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:56.592606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:39749 (local address 127.11.60.2:56325)
0811 03:11:56.602667 (+ 10061us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:56.602690 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:56.602821 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:56.606006 (+ 3185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:56.606013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:56.606777 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:56.607308 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:56.607324 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.609954 (+ 2630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.609960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:56.613295 (+ 3335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:56.613311 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.620203 (+ 6892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.620210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:56.623396 (+ 3186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:56.624003 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:56.624027 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:56.629527 (+ 5500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:56.633415 (+ 3888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:56.633421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:56.633425 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:56.633802 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:56.634191 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:56.634196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:56.634200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:56.634403 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:56.634916 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:56.634921 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:56.635081 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 03:11:56.635212 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 03:11:56.635556 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9895,"mutex_wait_us":25,"thread_start_us":10293,"threads_started":1}
I20260811 03:11:56.636037 19556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:56.585178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59202 (local address 127.11.60.3:42001)
0811 03:11:56.610917 (+ 25739us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:56.610927 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:56.610961 (+ 34us) server_negotiation.cc:407] Connection header received
0811 03:11:56.611064 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:56.611074 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:56.611371 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:56.611567 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:56.613759 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.616590 (+ 2831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.618505 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.619458 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.620636 (+ 1178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:56.628551 (+ 7915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:56.628569 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:56.628586 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:56.628706 (+ 120us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:56.632352 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:56.633355 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:56.633364 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:56.633366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:56.633544 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:56.634277 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:56.634281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:56.634283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:56.634703 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:56.635015 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:56.635376 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 03:11:56.635687 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25543,"thread_start_us":25384,"threads_started":1}
I20260811 03:11:56.636142 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:56.586806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:56325 (local address 127.11.60.60:39749)
0811 03:11:56.605222 (+ 18416us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:56.605237 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:56.605265 (+ 28us) server_negotiation.cc:407] Connection header received
0811 03:11:56.605358 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:56.605364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:56.605454 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:56.605592 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:56.607602 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.609764 (+ 2162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.613471 (+ 3707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:56.615979 (+ 2508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:56.623794 (+ 7815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:56.629809 (+ 6015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:56.629819 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:56.629824 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:56.629871 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:56.633225 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:56.633941 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:56.633947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:56.633950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:56.634078 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:56.634518 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:56.634527 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:56.634529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:56.634795 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:56.634948 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:56.635337 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 03:11:56.635566 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18210,"thread_start_us":20612,"threads_started":1}
I20260811 03:11:56.636904 19546 heartbeater.cc:347] Connected to a master server at 127.11.60.60:39749
I20260811 03:11:56.637045 19546 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:56.637506 19546 heartbeater.cc:511] Master 127.11.60.60:39749 requested a full tablet report, sending...
I20260811 03:11:56.637522 18821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:56.638886 19190 ts_manager.cc:194] Registered new tserver with Master: ee8e84046be747fc98b50003173b3342 (127.11.60.2:43797)
I20260811 03:11:56.724081 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 18681
W20260811 03:11:56.744189 18301 connection.cc:535] server connection from 127.11.60.3:54551 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 03:11:56.745162 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:42001
--local_ip_for_outbound_sockets=127.11.60.3
--tserver_master_addrs=127.11.60.62:42751
--webserver_port=33445
--webserver_interface=127.11.60.3
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab
--principal=oryx/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.60.62:42751,127.11.60.61:42747
--tserver_master_addrs=127.11.60.62:42751,127.11.60.61:42747,127.11.60.60:39749 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:11:56.976971 19381 heartbeater.cc:503] Master 127.11.60.62:42751 was elected leader, sending a full tablet report...
W20260811 03:11:57.100075 19564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:11:57.100412 19564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:11:57.100587 19564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:11:57.108227 19564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:11:57.108378 19564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:11:57.108587 19564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:11:57.118588 19564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab
--principal=oryx/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:42001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=33445
--tserver_master_addrs=127.11.60.62:42751,127.11.60.61:42747,127.11.60.60:39749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:11:57.120046 19564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:11:57.121420 19564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:11:57.137295 19570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:57.137332 19571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:11:57.137526 19573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:11:57.138396 19564 server_base.cc:1034] running on GCE node
I20260811 03:11:57.139102 19564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:11:57.140172 19564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:11:57.141423 19564 hybrid_clock.cc:648] HybridClock initialized: now 1786417917141361 us; error 50 us; skew 500 ppm
I20260811 03:11:57.141772 19564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:11:57 dist-test-slave-tm1g krb5kdc[18269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417917, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:11:57.145232 19564 init.cc:377] Logged in from keytab as oryx/127.11.60.3@KRBTEST.COM (short username oryx)
I20260811 03:11:57.146868 19564 webserver.cc:460] Webserver started at http://127.11.60.3:33445/ using document root <none> and password file <none>
I20260811 03:11:57.147707 19564 fs_manager.cc:359] Metadata directory not provided
I20260811 03:11:57.147871 19564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:11:57.153779 19564 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 03:11:57.156800 19580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:57.158458 19564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 03:11:57.158653 19564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "884a718d1b9b4654a00529bffe0ba706"
format_stamp: "Formatted at 2026-08-11 03:11:53 on dist-test-slave-tm1g"
server_key: "100266dd8994ea1eefd004a1760e276e"
server_key_iv: "ff074425a326ebc07ec5de5f4a28d38b"
server_key_version: "encryptionkey@0"
I20260811 03:11:57.159752 19564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:11:57.182133 19564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:11:57.187706 19564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:11:57.188036 19564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:11:57.189211 19564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:11:57.190718 19564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:11:57.190837 19564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:57.190969 19564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:11:57.191056 19564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:11:57.227819 19564 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:42001
I20260811 03:11:57.227905 19696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:42001 every 8 connection(s)
I20260811 03:11:57.229287 19564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:11:57.234570 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 19564
Aug 11 03:11:57 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417917, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:11:57.272362 19703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.232852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:39253 (local address 127.11.60.62:42751)
0811 03:11:57.245338 (+ 12486us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.245348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.245381 (+ 33us) server_negotiation.cc:407] Connection header received
0811 03:11:57.245471 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.245481 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.245618 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.245786 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:57.248955 (+ 3169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.250549 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.254370 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.255207 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.255375 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.263231 (+ 7856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:57.263239 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:57.263243 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:57.263288 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:57.267739 (+ 4451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.269201 (+ 1462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.269209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.269213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.269380 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.269933 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.269937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.269941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.270412 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:57.270557 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.271718 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.271966 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12261,"thread_start_us":13091,"threads_started":1}
I20260811 03:11:57.275344 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.233335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.3:39253)
0811 03:11:57.240464 (+ 7129us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:57.240508 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:57.242466 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:57.246202 (+ 3736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:57.246222 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:57.247297 (+ 1075us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:57.248535 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.248571 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.251606 (+ 3035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.251612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.254137 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.254161 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.255488 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.255499 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.255928 (+ 429us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.256963 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:57.257015 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:57.262788 (+ 5773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:57.267948 (+ 5160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:57.267964 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:57.267994 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:57.269059 (+ 1065us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:57.269486 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:57.269493 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:57.269499 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:57.269805 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:57.270554 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:57.270571 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:57.271220 (+ 649us) client_negotiation.cc:769] Sending connection context
0811 03:11:57.271619 (+ 399us) client_negotiation.cc:240] Negotiation successful
0811 03:11:57.273345 (+ 1726us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6834,"thread_start_us":6609,"threads_started":1}
Aug 11 03:11:57 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417917, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.61@KRBTEST.COM
Aug 11 03:11:57 dist-test-slave-tm1g krb5kdc[18269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417917, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.60@KRBTEST.COM
I20260811 03:11:57.285648 19699 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42751
I20260811 03:11:57.286072 19699 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:57.287177 19699 heartbeater.cc:511] Master 127.11.60.62:42751 requested a full tablet report, sending...
I20260811 03:11:57.289220 18315 ts_manager.cc:194] Re-registered known tserver with Master: 884a718d1b9b4654a00529bffe0ba706 (127.11.60.3:42001)
I20260811 03:11:57.290406 18315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:39253
I20260811 03:11:57.294791 19715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.253337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:39749 (local address 127.11.60.3:34213)
0811 03:11:57.263128 (+ 9791us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:57.263155 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:57.263284 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:57.266209 (+ 2925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:57.266219 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:57.268043 (+ 1824us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:57.268554 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.268572 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.270216 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.270222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.272895 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.272913 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.277127 (+ 4214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.277135 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.277365 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.279406 (+ 2041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:57.279435 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:57.282178 (+ 2743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:57.290952 (+ 8774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:57.290961 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:57.290968 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:57.291732 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:57.292335 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:57.292344 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:57.292350 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:57.292578 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:57.293518 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:57.293526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:57.293725 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 03:11:57.293888 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 03:11:57.294197 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9638,"mutex_wait_us":703,"thread_start_us":10017,"threads_started":1}
I20260811 03:11:57.295980 19710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.247238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:34213 (local address 127.11.60.60:39749)
0811 03:11:57.265648 (+ 18410us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.265656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.265679 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:11:57.265752 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.265762 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.265865 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.265987 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:57.268795 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.270047 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.273086 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.273950 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.274093 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.282464 (+ 8371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:57.282474 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:57.282479 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:57.282530 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:57.290730 (+ 8200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.291934 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.291944 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.291950 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.292160 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.292748 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.292756 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.292761 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.293257 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:57.293436 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.295297 (+ 1861us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.295579 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18271,"thread_start_us":18129,"threads_started":1}
I20260811 03:11:57.296728 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.232537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:36215 (local address 127.11.60.61:42747)
0811 03:11:57.233121 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.233131 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.247007 (+ 13876us) server_negotiation.cc:407] Connection header received
0811 03:11:57.247346 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.247355 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.247471 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.247581 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:57.250121 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.251415 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.274925 (+ 23510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.275932 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.276085 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.283484 (+ 7399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:57.283494 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:57.283500 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:57.283548 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:57.287418 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.288386 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.288392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.288395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.288538 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.294687 (+ 6149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.294693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.294696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.295162 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:57.295301 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.295975 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.296258 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":441,"threads_started":1}
I20260811 03:11:57.296854 19697 heartbeater.cc:347] Connected to a master server at 127.11.60.60:39749
I20260811 03:11:57.297016 19697 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:57.297380 19697 heartbeater.cc:511] Master 127.11.60.60:39749 requested a full tablet report, sending...
I20260811 03:11:57.298249 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.234081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42747 (local address 127.11.60.3:36215)
0811 03:11:57.246893 (+ 12812us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:57.246914 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:57.247171 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:57.247892 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:57.247900 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:57.248799 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:11:57.249869 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.249886 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.266137 (+ 16251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.266147 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.272471 (+ 6324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.272488 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.276441 (+ 3953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.276451 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.276691 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.277769 (+ 1078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:11:57.277797 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:11:57.281425 (+ 3628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:11:57.287595 (+ 6170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:57.287600 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:57.287604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:57.288217 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:57.292507 (+ 4290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:11:57.292512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:11:57.292516 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:11:57.292674 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:11:57.295412 (+ 2738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:11:57.295422 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:11:57.295571 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 03:11:57.296600 (+ 1029us) client_negotiation.cc:240] Negotiation successful
0811 03:11:57.297858 (+ 1258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12675,"thread_start_us":12435,"threads_started":1}
I20260811 03:11:57.298792 19190 ts_manager.cc:194] Registered new tserver with Master: 884a718d1b9b4654a00529bffe0ba706 (127.11.60.3:42001)
I20260811 03:11:57.301013 19700 heartbeater.cc:347] Connected to a master server at 127.11.60.61:42747
I20260811 03:11:57.301172 19700 heartbeater.cc:464] Registering TS with master...
I20260811 03:11:57.301579 19700 heartbeater.cc:511] Master 127.11.60.61:42747 requested a full tablet report, sending...
I20260811 03:11:57.304064 18985 ts_manager.cc:194] Registered new tserver with Master: 884a718d1b9b4654a00529bffe0ba706 (127.11.60.3:42001)
I20260811 03:11:57.313099 19717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.255719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40338 (local address 127.11.60.61:42747)
0811 03:11:57.278132 (+ 22413us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.278139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.279045 (+ 906us) server_negotiation.cc:407] Connection header received
0811 03:11:57.279118 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.279126 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.279238 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.280136 (+ 898us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:57.288663 (+ 8527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.290516 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.300530 (+ 10014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.301181 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.301337 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.305805 (+ 4468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:57.305812 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:57.305819 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:57.305862 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:57.309910 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.310810 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.310817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.310824 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.311002 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.311391 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.311396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.311399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.311804 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:57.312047 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.312411 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.312631 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22286,"thread_start_us":22125,"threads_started":1}
I20260811 03:11:57.314450 19718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.289687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50586 (local address 127.11.60.60:39749)
0811 03:11:57.294158 (+ 4471us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.294166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.295136 (+ 970us) server_negotiation.cc:407] Connection header received
0811 03:11:57.295385 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.295390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.295469 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.295579 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:57.297867 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.299196 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.301241 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.301992 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.302104 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.307209 (+ 5105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:57.307219 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:57.307224 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:57.307273 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:57.310741 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.311515 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.311520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.311523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.311680 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.312167 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.312173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.312176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.312537 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:57.312774 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.313782 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.314031 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4321,"thread_start_us":4155,"threads_started":1}
I20260811 03:11:57.315999 19703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.254760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57354 (local address 127.11.60.62:42751)
0811 03:11:57.272646 (+ 17886us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.272651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.289645 (+ 16994us) server_negotiation.cc:407] Connection header received
0811 03:11:57.289819 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.289824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.289927 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.290026 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:57.299315 (+ 9289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.300136 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.304204 (+ 4068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.304817 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.304968 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.308273 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:57.308278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:57.308302 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:57.308351 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:57.311828 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.312683 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.312689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.312693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.312847 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.313309 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.313314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.313318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.314536 (+ 1218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:57.314790 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.315341 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.315638 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17759,"thread_start_us":26773,"threads_started":1}
I20260811 03:11:57.320148 18315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:11:57.328091 18315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:11:57.360553 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.350186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36860 (local address 127.11.60.1:39959)
0811 03:11:57.350806 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.350815 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.350845 (+ 30us) server_negotiation.cc:407] Connection header received
0811 03:11:57.350955 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.350964 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.351267 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.351510 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:57.353630 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.355055 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.357936 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.359471 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.359616 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.359862 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.360083 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.360254 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":334,"threads_started":1}
I20260811 03:11:57.360545 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.349896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:39959 (local address 127.0.0.1:36860)
0811 03:11:57.350527 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:57.350544 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:57.350645 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:57.351498 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:57.351503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:57.351528 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:57.353394 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.353408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.355247 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.355253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.357768 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.357781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.359673 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.359679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.359873 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.359920 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 03:11:57.360035 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 03:11:57.360169 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":353,"threads_started":1}
I20260811 03:11:57.363655 19736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.351325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:42001 (local address 127.0.0.1:59210)
0811 03:11:57.352665 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:57.352685 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:57.352805 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:57.354435 (+ 1630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:57.354442 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:57.354490 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:57.355069 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.355084 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.357534 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.357542 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.360063 (+ 2521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.360078 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.362929 (+ 2851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.362935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.363140 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.363178 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 03:11:57.363249 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 03:11:57.363352 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1206,"thread_start_us":1575,"threads_started":1}
I20260811 03:11:57.363834 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.351916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.11.60.3:42001)
0811 03:11:57.353781 (+ 1865us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.353791 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.353827 (+ 36us) server_negotiation.cc:407] Connection header received
0811 03:11:57.353924 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.353931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.354178 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.354424 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:57.355280 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.357329 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.360940 (+ 3611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.362718 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.362863 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.363108 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.363317 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.363500 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1684,"thread_start_us":809,"threads_started":1}
I20260811 03:11:57.365175 19311 tablet_service.cc:1467] Processing CreateTablet for tablet ae679dc15a704279adb5f5a2c2d424cb (DEFAULT_TABLE table=test-table [id=c17a890fad43447289eed27c85270740]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:57.366928 19311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae679dc15a704279adb5f5a2c2d424cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:57.367892 19630 tablet_service.cc:1467] Processing CreateTablet for tablet ae679dc15a704279adb5f5a2c2d424cb (DEFAULT_TABLE table=test-table [id=c17a890fad43447289eed27c85270740]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:57.369370 19630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae679dc15a704279adb5f5a2c2d424cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:57.374965 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.352883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48790 (local address 127.11.60.2:43797)
0811 03:11:57.353604 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.353617 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.359124 (+ 5507us) server_negotiation.cc:407] Connection header received
0811 03:11:57.359306 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.359315 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.359566 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.359764 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:57.361821 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.362960 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.372793 (+ 9833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.373789 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.373957 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.374188 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.374505 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.374666 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":366,"threads_started":1}
I20260811 03:11:57.374997 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.352585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:43797 (local address 127.0.0.1:48790)
0811 03:11:57.358966 (+ 6381us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:57.358987 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:57.359119 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:57.361021 (+ 1902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:57.361028 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:57.361056 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:57.361603 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.361619 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.370436 (+ 8817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.370442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.372533 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.372553 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.374021 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.374028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.374224 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.374270 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 03:11:57.374433 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 03:11:57.374595 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6235,"thread_start_us":6070,"threads_started":1}
I20260811 03:11:57.378382 19469 tablet_service.cc:1467] Processing CreateTablet for tablet ae679dc15a704279adb5f5a2c2d424cb (DEFAULT_TABLE table=test-table [id=c17a890fad43447289eed27c85270740]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:11:57.379809 19469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae679dc15a704279adb5f5a2c2d424cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:11:57.384264 19739 tablet_bootstrap.cc:492] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: Bootstrap starting.
I20260811 03:11:57.387297 19740 tablet_bootstrap.cc:492] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: Bootstrap starting.
I20260811 03:11:57.389976 19739 tablet_bootstrap.cc:654] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:57.390139 19740 tablet_bootstrap.cc:654] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:57.391201 19740 log.cc:824] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:57.391660 19739 log.cc:824] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:57.394421 19739 tablet_bootstrap.cc:492] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: No bootstrap required, opened a new log
I20260811 03:11:57.394512 19740 tablet_bootstrap.cc:492] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: No bootstrap required, opened a new log
I20260811 03:11:57.394742 19739 ts_tablet_manager.cc:1397] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260811 03:11:57.394892 19740 ts_tablet_manager.cc:1397] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 03:11:57.395447 19745 tablet_bootstrap.cc:492] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: Bootstrap starting.
I20260811 03:11:57.399484 19745 tablet_bootstrap.cc:654] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: Neither blocks nor log segments found. Creating new log.
I20260811 03:11:57.400920 19745 log.cc:824] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: Log is configured to *not* fsync() on all Append() calls
I20260811 03:11:57.404074 19740 raft_consensus.cc:348] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } } peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } }
I20260811 03:11:57.404500 19745 tablet_bootstrap.cc:492] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: No bootstrap required, opened a new log
I20260811 03:11:57.404525 19740 raft_consensus.cc:374] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:57.404673 19740 raft_consensus.cc:729] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884a718d1b9b4654a00529bffe0ba706, State: Initialized, Role: FOLLOWER
I20260811 03:11:57.404799 19745 ts_tablet_manager.cc:1397] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 03:11:57.405378 19740 consensus_queue.cc:260] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } } peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } }
I20260811 03:11:57.405750 19739 raft_consensus.cc:348] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } } peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } }
I20260811 03:11:57.406258 19739 raft_consensus.cc:374] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:57.406419 19739 raft_consensus.cc:729] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cfcb8537b49453096cea3274246b3d2, State: Initialized, Role: FOLLOWER
I20260811 03:11:57.407084 19739 consensus_queue.cc:260] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } } peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } }
I20260811 03:11:57.407603 19740 ts_tablet_manager.cc:1428] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260811 03:11:57.407882 19699 heartbeater.cc:503] Master 127.11.60.62:42751 was elected leader, sending a full tablet report...
I20260811 03:11:57.416553 19739 ts_tablet_manager.cc:1428] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: Time spent starting tablet: real 0.022s user 0.011s sys 0.009s
I20260811 03:11:57.417984 19745 raft_consensus.cc:348] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } } peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } }
I20260811 03:11:57.418490 19745 raft_consensus.cc:374] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:11:57.418601 19745 raft_consensus.cc:729] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8e84046be747fc98b50003173b3342, State: Initialized, Role: FOLLOWER
I20260811 03:11:57.419278 19745 consensus_queue.cc:260] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } } peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } }
I20260811 03:11:57.420953 19547 heartbeater.cc:503] Master 127.11.60.62:42751 was elected leader, sending a full tablet report...
I20260811 03:11:57.421710 19745 ts_tablet_manager.cc:1428] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: Time spent starting tablet: real 0.017s user 0.008s sys 0.010s
W20260811 03:11:57.439105 19383 tablet.cc:2369] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:11:57.488483 19701 tablet.cc:2369] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:11:57.596506 19549 tablet.cc:2369] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:11:57.707391 19747 raft_consensus.cc:482] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:11:57.707772 19747 raft_consensus.cc:504] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } } peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } }
I20260811 03:11:57.709833 19747 leader_election.cc:288] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cfcb8537b49453096cea3274246b3d2 (127.11.60.1:39959), ee8e84046be747fc98b50003173b3342 (127.11.60.2:43797)
I20260811 03:11:57.715700 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.710374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:38111 (local address 127.11.60.1:39959)
0811 03:11:57.710624 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.710631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.710650 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:11:57.710728 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.710732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.710810 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.710905 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:57.711300 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.712074 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.713889 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.714757 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.714894 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.715004 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.715274 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.715405 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 03:11:57.715739 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.710088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:39959 (local address 127.11.60.3:38111)
0811 03:11:57.710322 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:57.710347 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:57.710499 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:57.710936 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:57.710941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:57.710969 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:57.711160 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.711168 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.712227 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.712233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.713724 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.713738 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.714926 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.714933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.715116 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.715141 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 03:11:57.715257 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 03:11:57.715405 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":80}
I20260811 03:11:57.716157 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.710744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:45601 (local address 127.11.60.2:43797)
0811 03:11:57.710979 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.710987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.711007 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:11:57.711074 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.711078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.711133 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.711230 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:57.711632 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.712358 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.714070 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.715153 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.715305 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.715407 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.715746 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.715890 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 03:11:57.716157 19715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.710530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:43797 (local address 127.11.60.3:45601)
0811 03:11:57.710751 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:57.710766 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:57.710895 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:57.711254 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:57.711258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:57.711284 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:57.711490 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.711498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.712515 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.712521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.713900 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:57.713913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.715335 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.715342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:57.715577 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.715605 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 03:11:57.715716 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 03:11:57.715868 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":25}
I20260811 03:11:57.717262 19333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae679dc15a704279adb5f5a2c2d424cb" candidate_uuid: "884a718d1b9b4654a00529bffe0ba706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cfcb8537b49453096cea3274246b3d2" is_pre_election: true
I20260811 03:11:57.717264 19495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae679dc15a704279adb5f5a2c2d424cb" candidate_uuid: "884a718d1b9b4654a00529bffe0ba706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee8e84046be747fc98b50003173b3342" is_pre_election: true
I20260811 03:11:57.717923 19495 raft_consensus.cc:2454] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884a718d1b9b4654a00529bffe0ba706 in term 0.
I20260811 03:11:57.717864 19333 raft_consensus.cc:2454] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884a718d1b9b4654a00529bffe0ba706 in term 0.
I20260811 03:11:57.718684 19584 leader_election.cc:302] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884a718d1b9b4654a00529bffe0ba706, 9cfcb8537b49453096cea3274246b3d2; no voters:
I20260811 03:11:57.719106 19747 raft_consensus.cc:2781] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:11:57.719281 19747 raft_consensus.cc:482] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:11:57.719400 19747 raft_consensus.cc:3037] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:57.721766 19747 raft_consensus.cc:504] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } } peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } }
I20260811 03:11:57.722467 19747 leader_election.cc:288] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [CANDIDATE]: Term 1 election: Requested vote from peers 9cfcb8537b49453096cea3274246b3d2 (127.11.60.1:39959), ee8e84046be747fc98b50003173b3342 (127.11.60.2:43797)
I20260811 03:11:57.723067 19333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae679dc15a704279adb5f5a2c2d424cb" candidate_uuid: "884a718d1b9b4654a00529bffe0ba706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cfcb8537b49453096cea3274246b3d2"
I20260811 03:11:57.723158 19495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae679dc15a704279adb5f5a2c2d424cb" candidate_uuid: "884a718d1b9b4654a00529bffe0ba706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee8e84046be747fc98b50003173b3342"
I20260811 03:11:57.723282 19333 raft_consensus.cc:3037] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:57.723460 19495 raft_consensus.cc:3037] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:11:57.725786 19333 raft_consensus.cc:2454] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884a718d1b9b4654a00529bffe0ba706 in term 1.
I20260811 03:11:57.725956 19495 raft_consensus.cc:2454] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884a718d1b9b4654a00529bffe0ba706 in term 1.
I20260811 03:11:57.726358 19584 leader_election.cc:302] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884a718d1b9b4654a00529bffe0ba706, 9cfcb8537b49453096cea3274246b3d2; no voters:
I20260811 03:11:57.726761 19747 raft_consensus.cc:2781] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:11:57.727279 19747 raft_consensus.cc:686] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 1 LEADER]: Becoming Leader. State: Replica: 884a718d1b9b4654a00529bffe0ba706, State: Running, Role: LEADER
I20260811 03:11:57.727874 19747 consensus_queue.cc:237] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } } peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } }
I20260811 03:11:57.733904 18315 catalog_manager.cc:5495] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 reported cstate change: term changed from 0 to 1, leader changed from <none> to 884a718d1b9b4654a00529bffe0ba706 (127.11.60.3). New cstate: current_term: 1 leader_uuid: "884a718d1b9b4654a00529bffe0ba706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } health_report { overall_health: UNKNOWN } } }
I20260811 03:11:57.820509 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.814675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.11.60.3:42001)
0811 03:11:57.814880 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.814885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.814912 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:11:57.815093 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.815097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.815196 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.815369 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:11:57.815783 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.816586 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.818027 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.818586 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.818695 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.819187 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:11:57.819806 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:11:57.819928 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.820089 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.820165 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 03:11:57.829632 19333 raft_consensus.cc:1261] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2 [term 1 FOLLOWER]: Refusing update from remote peer 884a718d1b9b4654a00529bffe0ba706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:57.829623 19495 raft_consensus.cc:1261] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342 [term 1 FOLLOWER]: Refusing update from remote peer 884a718d1b9b4654a00529bffe0ba706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:11:57.830821 19750 consensus_queue.cc:1035] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:57.831214 19747 consensus_queue.cc:1035] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:11:57.856582 19756 mvcc.cc:205] Tried to move back new op lower bound from 7317167791417982976 to 7317167791017930752. Current Snapshot: MvccSnapshot[applied={T|T < 7317167791417982976}]
I20260811 03:11:57.893432 18314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:57.893726 18314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:11:57.900977 18314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 332214273aae4803b3f58ceca1e2b794: Sending DeleteTablet for 3 replicas of tablet ae679dc15a704279adb5f5a2c2d424cb
I20260811 03:11:57.902346 19311 tablet_service.cc:1514] Processing DeleteTablet for tablet ae679dc15a704279adb5f5a2c2d424cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:57 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:36860
I20260811 03:11:57.903746 19630 tablet_service.cc:1514] Processing DeleteTablet for tablet ae679dc15a704279adb5f5a2c2d424cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:57 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:59210
I20260811 03:11:57.903780 19469 tablet_service.cc:1514] Processing DeleteTablet for tablet ae679dc15a704279adb5f5a2c2d424cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:11:57 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:48790
I20260811 03:11:57.910638 19764 tablet_replica.cc:330] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: stopping tablet replica
I20260811 03:11:57.911242 19764 raft_consensus.cc:2229] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:57.911862 19764 raft_consensus.cc:2258] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:57.915097 19766 tablet_replica.cc:330] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: stopping tablet replica
I20260811 03:11:57.915723 19766 raft_consensus.cc:2229] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:11:57.916244 19764 ts_tablet_manager.cc:1905] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:57.916322 19766 raft_consensus.cc:2258] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:57.917047 19765 tablet_replica.cc:330] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: stopping tablet replica
I20260811 03:11:57.917860 19765 raft_consensus.cc:2229] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:11:57.918708 19765 raft_consensus.cc:2258] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:11:57.919529 19766 ts_tablet_manager.cc:1905] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:57.920802 19765 ts_tablet_manager.cc:1905] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:11:57.925700 19764 ts_tablet_manager.cc:1918] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:57.925925 19764 log.cc:1196] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/ae679dc15a704279adb5f5a2c2d424cb
I20260811 03:11:57.926591 19764 ts_tablet_manager.cc:1939] T ae679dc15a704279adb5f5a2c2d424cb P 9cfcb8537b49453096cea3274246b3d2: Deleting consensus metadata
I20260811 03:11:57.927690 19766 ts_tablet_manager.cc:1918] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:57.927891 19766 log.cc:1196] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/ae679dc15a704279adb5f5a2c2d424cb
I20260811 03:11:57.928558 19766 ts_tablet_manager.cc:1939] T ae679dc15a704279adb5f5a2c2d424cb P ee8e84046be747fc98b50003173b3342: Deleting consensus metadata
I20260811 03:11:57.928874 18301 catalog_manager.cc:4841] TS 9cfcb8537b49453096cea3274246b3d2 (127.11.60.1:39959): tablet ae679dc15a704279adb5f5a2c2d424cb (table test-table [id=c17a890fad43447289eed27c85270740]) successfully deleted
I20260811 03:11:57.929664 19765 ts_tablet_manager.cc:1918] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:11:57.929860 19765 log.cc:1196] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/ae679dc15a704279adb5f5a2c2d424cb
I20260811 03:11:57.930508 19765 ts_tablet_manager.cc:1939] T ae679dc15a704279adb5f5a2c2d424cb P 884a718d1b9b4654a00529bffe0ba706: Deleting consensus metadata
I20260811 03:11:57.932753 18300 catalog_manager.cc:4841] TS 884a718d1b9b4654a00529bffe0ba706 (127.11.60.3:42001): tablet ae679dc15a704279adb5f5a2c2d424cb (table test-table [id=c17a890fad43447289eed27c85270740]) successfully deleted
I20260811 03:11:57.935492 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 18282
I20260811 03:11:57.936734 18351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:11:57.952862 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.907403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40350 (local address 127.11.60.61:42747)
0811 03:11:57.930051 (+ 22648us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.930056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.931077 (+ 1021us) server_negotiation.cc:407] Connection header received
0811 03:11:57.931318 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.931324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.931426 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.931535 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:57.935018 (+ 3483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.936805 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.939300 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.939988 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.940373 (+ 385us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.944363 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:57.944372 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:57.944376 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:57.944417 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:57.949041 (+ 4624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.950018 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.950024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.950028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.950216 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.950921 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.950926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.950931 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.951333 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:57.951600 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.952200 (+ 600us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.952457 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22431,"thread_start_us":22651,"threads_started":1}
I20260811 03:11:57.975378 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:57.931905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.11.60.60:39749)
0811 03:11:57.935424 (+ 3519us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:57.935431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:57.953983 (+ 18552us) server_negotiation.cc:407] Connection header received
0811 03:11:57.954281 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:57.954288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:57.954399 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:57.954487 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:57.956902 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.958093 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.961188 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:57.962110 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:57.962226 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:57.967733 (+ 5507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:57.967741 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:57.967746 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:57.967795 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:57.971859 (+ 4064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.972964 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.972970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.972974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.973156 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:57.973659 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:57.973665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:57.973670 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:57.974059 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:57.974353 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:57.974737 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 03:11:57.974992 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3335,"thread_start_us":4067,"threads_started":1}
I20260811 03:11:58.087203 18301 catalog_manager.cc:4841] TS ee8e84046be747fc98b50003173b3342 (127.11.60.2:43797): tablet ae679dc15a704279adb5f5a2c2d424cb (table test-table [id=c17a890fad43447289eed27c85270740]) successfully deleted
W20260811 03:11:58.087560 18301 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.60.2:43797, user_credentials={real_user=slave}} blocked reactor thread for 139891us
I20260811 03:11:58.087903 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 18282
I20260811 03:11:58.138515 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:58.120737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40360 (local address 127.11.60.61:42747)
0811 03:11:58.121016 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:58.121027 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:58.122248 (+ 1221us) server_negotiation.cc:407] Connection header received
0811 03:11:58.122687 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:58.122694 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:58.122789 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:58.122913 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:58.124572 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:58.125732 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:58.127274 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:58.127920 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:58.128071 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:58.131618 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:58.131626 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:58.131634 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:58.131685 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:58.135250 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:58.136201 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:58.136206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:58.136210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:58.136373 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:58.136814 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:58.136819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:58.136824 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:58.137217 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:58.137509 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:58.137862 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 03:11:58.138140 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 03:11:58.140051 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:58.122258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50606 (local address 127.11.60.60:39749)
0811 03:11:58.122496 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:58.122502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:58.122524 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:11:58.122750 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:58.122760 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:58.122839 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:58.122953 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:11:58.126042 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:58.127321 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:58.129084 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:58.129621 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:58.129754 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:58.133243 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:11:58.133249 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:11:58.133254 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:11:58.133313 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:11:58.137092 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:58.137858 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:58.137864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:58.137867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:58.138025 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:11:58.138490 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:11:58.138496 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:11:58.138502 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:11:58.138896 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:11:58.139154 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:58.139522 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 03:11:58.139763 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 03:11:58.866680 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:58.865773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.3:37961)
0811 03:11:58.866358 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:58.866506 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":267,"threads_started":1}
W20260811 03:11:58.867223 19699 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:42751 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
I20260811 03:11:58.876039 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:58.874999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.2:43721)
0811 03:11:58.875638 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:58.875814 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":339,"threads_started":1}
W20260811 03:11:58.876463 19547 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:42751 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
I20260811 03:11:58.883354 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:58.882288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.1:35023)
0811 03:11:58.882959 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:58.883107 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":423,"threads_started":1}
W20260811 03:11:58.883790 19381 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:42751 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
I20260811 03:11:59.401917 19781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 332214273aae4803b3f58ceca1e2b794)
I20260811 03:11:59.402271 19782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 332214273aae4803b3f58ceca1e2b794)
I20260811 03:11:59.402266 19781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } }
I20260811 03:11:59.402585 19782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } }
I20260811 03:11:59.403915 19782 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751), 80fe5eac26de4645843c11ce986878de (127.11.60.60:39749)
I20260811 03:11:59.403903 19781 leader_election.cc:288] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751), 6f2b972980a94c26b2d8932f832ed983 (127.11.60.61:42747)
I20260811 03:11:59.412117 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.404056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35350)
0811 03:11:59.411547 (+ 7491us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.411767 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7301,"threads_started":1}
I20260811 03:11:59.416965 19785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.404060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35338)
0811 03:11:59.416434 (+ 12374us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.416665 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12204,"threads_started":1}
W20260811 03:11:59.416952 18970 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
W20260811 03:11:59.418197 19177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
I20260811 03:11:59.419209 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.418499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35356)
0811 03:11:59.418670 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.418889 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 03:11:59.419405 18970 leader_election.cc:334] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751): Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
I20260811 03:11:59.420769 19785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.420362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35362)
0811 03:11:59.420527 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.420627 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 03:11:59.421006 19177 leader_election.cc:334] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751): Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
I20260811 03:11:59.424695 19787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.412695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53896 (local address 127.11.60.61:42747)
0811 03:11:59.416962 (+ 4267us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:59.416969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:59.417259 (+ 290us) server_negotiation.cc:407] Connection header received
0811 03:11:59.417511 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:59.417519 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:59.417618 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:59.417758 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:59.418702 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:59.419993 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:59.422126 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:59.423685 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:59.423830 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:59.423929 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:59.424201 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 03:11:59.424326 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4120,"thread_start_us":4218,"threads_started":1}
I20260811 03:11:59.424736 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.404353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42747 (local address 127.0.0.1:53896)
0811 03:11:59.417130 (+ 12777us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.417154 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:59.417308 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:59.418008 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:59.418016 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:59.418056 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:59.418470 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:59.418485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:59.420137 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:59.420142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:59.421925 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:59.421938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:59.423859 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:59.423865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:59.424043 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:59.424067 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:11:59.424166 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 03:11:59.424304 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12628,"mutex_wait_us":3,"thread_start_us":11733,"threads_started":1}
I20260811 03:11:59.425575 19789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.417422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53620 (local address 127.11.60.60:39749)
0811 03:11:59.418644 (+ 1222us) server_negotiation.cc:206] Beginning negotiation
0811 03:11:59.418651 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:11:59.418672 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:11:59.418731 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:11:59.418735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:11:59.418809 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:11:59.418914 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:11:59.419694 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:59.420993 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:59.423666 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:11:59.424523 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:11:59.424675 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:59.424804 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 03:11:59.425091 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 03:11:59.425227 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":682,"thread_start_us":542,"threads_started":1}
I20260811 03:11:59.425554 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.404714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:39749 (local address 127.0.0.1:53620)
0811 03:11:59.413914 (+ 9200us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.413940 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:11:59.414121 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:11:59.418936 (+ 4815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:11:59.418941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:11:59.418980 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:11:59.419265 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:59.419275 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:59.421118 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:59.421125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:59.423518 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:11:59.423530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:11:59.424721 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:11:59.424728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:11:59.424908 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:11:59.424930 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 03:11:59.425033 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 03:11:59.425192 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8841,"thread_start_us":7601,"threads_started":1}
I20260811 03:11:59.425563 18996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80fe5eac26de4645843c11ce986878de" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f2b972980a94c26b2d8932f832ed983" is_pre_election: true
I20260811 03:11:59.425974 18996 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80fe5eac26de4645843c11ce986878de in term 1.
I20260811 03:11:59.426074 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f2b972980a94c26b2d8932f832ed983" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "80fe5eac26de4645843c11ce986878de" is_pre_election: true
I20260811 03:11:59.426445 19200 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f2b972980a94c26b2d8932f832ed983 in term 1.
I20260811 03:11:59.426484 19174 leader_election.cc:302] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f2b972980a94c26b2d8932f832ed983, 80fe5eac26de4645843c11ce986878de; no voters: 332214273aae4803b3f58ceca1e2b794
I20260811 03:11:59.426851 19781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 03:11:59.426975 19781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 FOLLOWER]: Starting leader election (detected failure of leader 332214273aae4803b3f58ceca1e2b794)
I20260811 03:11:59.426960 18967 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f2b972980a94c26b2d8932f832ed983, 80fe5eac26de4645843c11ce986878de; no voters: 332214273aae4803b3f58ceca1e2b794
I20260811 03:11:59.427081 19781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 1 FOLLOWER]: Advancing to term 2
I20260811 03:11:59.427346 19782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 03:11:59.427469 19782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 332214273aae4803b3f58ceca1e2b794)
I20260811 03:11:59.427572 19782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 1 FOLLOWER]: Advancing to term 2
I20260811 03:11:59.431912 19781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } }
I20260811 03:11:59.431998 19782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } }
I20260811 03:11:59.432659 19781 leader_election.cc:288] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [CANDIDATE]: Term 2 election: Requested vote from peers 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751), 6f2b972980a94c26b2d8932f832ed983 (127.11.60.61:42747)
I20260811 03:11:59.432754 19782 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 2 election: Requested vote from peers 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751), 80fe5eac26de4645843c11ce986878de (127.11.60.60:39749)
I20260811 03:11:59.433300 18996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80fe5eac26de4645843c11ce986878de" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f2b972980a94c26b2d8932f832ed983"
I20260811 03:11:59.433375 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.432742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35374)
0811 03:11:59.433123 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.433217 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267}
I20260811 03:11:59.433336 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f2b972980a94c26b2d8932f832ed983" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "80fe5eac26de4645843c11ce986878de"
I20260811 03:11:59.433578 18996 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 80fe5eac26de4645843c11ce986878de in current term 2: Already voted for candidate 6f2b972980a94c26b2d8932f832ed983 in this term.
I20260811 03:11:59.433609 19200 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f2b972980a94c26b2d8932f832ed983 in current term 2: Already voted for candidate 80fe5eac26de4645843c11ce986878de in this term.
I20260811 03:11:59.433857 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.432832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35380)
0811 03:11:59.433602 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.433691 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":645}
I20260811 03:11:59.434793 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.434422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35388)
0811 03:11:59.434545 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.434632 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 03:11:59.435037 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.434606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35390)
0811 03:11:59.434783 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.434873 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 03:11:59.435213 18970 leader_election.cc:334] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751): Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
W20260811 03:11:59.435247 19177 leader_election.cc:334] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751): Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
I20260811 03:11:59.435341 18970 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f2b972980a94c26b2d8932f832ed983; no voters: 332214273aae4803b3f58ceca1e2b794, 80fe5eac26de4645843c11ce986878de
I20260811 03:11:59.435372 19177 leader_election.cc:302] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80fe5eac26de4645843c11ce986878de; no voters: 332214273aae4803b3f58ceca1e2b794, 6f2b972980a94c26b2d8932f832ed983
I20260811 03:11:59.435591 19782 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 03:11:59.435657 19781 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 03:11:59.868891 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.868043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.3:50885)
0811 03:11:59.868606 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.868703 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":278,"threads_started":1}
I20260811 03:11:59.878047 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.877158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.2:45079)
0811 03:11:59.877766 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.877852 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":384,"threads_started":1}
I20260811 03:11:59.885386 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:11:59.884448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.1:38637)
0811 03:11:59.884997 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 03:11:59.885152 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":285,"threads_started":1}
I20260811 03:12:00.870807 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:00.869860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.3:42879)
0811 03:12:00.870452 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:00.870569 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":282,"threads_started":1}
I20260811 03:12:00.879768 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:00.878958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.2:33093)
0811 03:12:00.879519 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:00.879606 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":321,"threads_started":1}
I20260811 03:12:00.887353 19799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:00.886379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.1:56449)
0811 03:12:00.886967 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:00.887078 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":314,"threads_started":1}
I20260811 03:12:01.028992 19800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:12:01.029234 19800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } }
I20260811 03:12:01.029912 19800 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751), 80fe5eac26de4645843c11ce986878de (127.11.60.60:39749)
I20260811 03:12:01.030591 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f2b972980a94c26b2d8932f832ed983" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "80fe5eac26de4645843c11ce986878de" is_pre_election: true
I20260811 03:12:01.030903 19200 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f2b972980a94c26b2d8932f832ed983 in term 2.
I20260811 03:12:01.031075 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.030086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35612)
0811 03:12:01.030651 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.030789 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":289,"threads_started":1}
I20260811 03:12:01.031376 18967 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f2b972980a94c26b2d8932f832ed983, 80fe5eac26de4645843c11ce986878de; no voters:
I20260811 03:12:01.031702 19800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 03:12:01.031867 19800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:12:01.032027 19800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 2 FOLLOWER]: Advancing to term 3
I20260811 03:12:01.032603 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.032275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35614)
0811 03:12:01.032406 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.032479 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 03:12:01.032859 18970 leader_election.cc:334] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751): Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
I20260811 03:12:01.034628 19800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } }
I20260811 03:12:01.035252 19800 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 3 election: Requested vote from peers 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751), 80fe5eac26de4645843c11ce986878de (127.11.60.60:39749)
I20260811 03:12:01.035732 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.035353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35628)
0811 03:12:01.035483 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.035563 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 03:12:01.035800 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f2b972980a94c26b2d8932f832ed983" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "80fe5eac26de4645843c11ce986878de"
I20260811 03:12:01.036072 19200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 2 FOLLOWER]: Advancing to term 3
I20260811 03:12:01.036780 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.036413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35644)
0811 03:12:01.036566 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.036647 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 03:12:01.036952 18970 leader_election.cc:334] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751): Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
I20260811 03:12:01.038524 19200 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f2b972980a94c26b2d8932f832ed983 in term 3.
I20260811 03:12:01.038923 18967 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f2b972980a94c26b2d8932f832ed983, 80fe5eac26de4645843c11ce986878de; no voters: 332214273aae4803b3f58ceca1e2b794
I20260811 03:12:01.039193 19800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 03:12:01.039597 19800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [term 3 LEADER]: Becoming Leader. State: Replica: 6f2b972980a94c26b2d8932f832ed983, State: Running, Role: LEADER
I20260811 03:12:01.039970 19800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } }
I20260811 03:12:01.041826 19803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2b972980a94c26b2d8932f832ed983. Latest consensus state: current_term: 3 leader_uuid: "6f2b972980a94c26b2d8932f832ed983" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:12:01.042080 19803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:01.042795 19805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:12:01.047035 19805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:12:01.047972 19805 catalog_manager.cc:1259] Loaded cluster ID: 9d125812b9264ea99502cb768c149147
I20260811 03:12:01.048092 19805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:12:01.049369 19805 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:12:01.050218 19805 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Loaded TSK: 0
I20260811 03:12:01.051167 19805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:12:01.105569 18985 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:12:01.112877 18985 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:12:01.117237 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.116683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35664)
0811 03:12:01.116952 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.117056 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 03:12:01.117311 19200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [term 3 FOLLOWER]: Refusing update from remote peer 6f2b972980a94c26b2d8932f832ed983: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 03:12:01.118017 19803 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 03:12:01.118273 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.117924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35670)
0811 03:12:01.118075 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.118145 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 03:12:01.118463 18970 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 -> Peer 332214273aae4803b3f58ceca1e2b794 (127.11.60.62:42751): Couldn't send request to peer 332214273aae4803b3f58ceca1e2b794. Status: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 03:12:01.121421 19807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2b972980a94c26b2d8932f832ed983. Latest consensus state: current_term: 3 leader_uuid: "6f2b972980a94c26b2d8932f832ed983" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:12:01.121754 19807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:12:01.122530 19803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6f2b972980a94c26b2d8932f832ed983" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:12:01.122848 19803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:01.123369 19807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6f2b972980a94c26b2d8932f832ed983" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "332214273aae4803b3f58ceca1e2b794" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42751 } } peers { permanent_uuid: "6f2b972980a94c26b2d8932f832ed983" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42747 } attrs { promote: false } } peers { permanent_uuid: "80fe5eac26de4645843c11ce986878de" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 39749 } attrs { promote: false } } }
I20260811 03:12:01.123675 19807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80fe5eac26de4645843c11ce986878de [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:12:01.123730 19808 mvcc.cc:205] Tried to move back new op lower bound from 7317167804889325568 to 7317167804583161856. Current Snapshot: MvccSnapshot[applied={T|T < 7317167804889325568}]
I20260811 03:12:01.150996 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.132311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43038 (local address 127.11.60.3:42001)
0811 03:12:01.142906 (+ 10595us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:01.142916 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:01.142942 (+ 26us) server_negotiation.cc:407] Connection header received
0811 03:12:01.143038 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:01.143043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:01.143125 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:01.143877 (+ 752us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:01.144162 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.145504 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.148539 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.149815 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.149951 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.150073 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:01.150438 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 03:12:01.150626 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10403,"thread_start_us":9871,"threads_started":1}
I20260811 03:12:01.151006 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.132199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:42001 (local address 127.0.0.1:43038)
0811 03:12:01.132471 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.132512 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:01.132669 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:01.143308 (+ 10639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:01.143313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:01.143344 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:01.143683 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:01.143701 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.145709 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.145717 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:01.148352 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:01.148365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.150004 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.150010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:01.150211 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.150249 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 03:12:01.150403 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 03:12:01.150576 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":87}
I20260811 03:12:01.152064 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.133052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:43797 (local address 127.0.0.1:46452)
0811 03:12:01.143264 (+ 10212us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.143285 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:01.143437 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:01.143828 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:01.143833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:01.143860 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:01.144235 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:01.144250 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.146117 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.146130 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:01.149401 (+ 3271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:01.149412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.151162 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.151168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:01.151350 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.151386 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 03:12:01.151481 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 03:12:01.151634 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10070,"spinlock_wait_cycles":2816,"thread_start_us":9909,"threads_started":1}
I20260811 03:12:01.151953 19630 tablet_service.cc:1467] Processing CreateTablet for tablet 3782773bfef54532b8db4121073fa27f (DEFAULT_TABLE table=test-table [id=7e869b6245e84b0c8717a0ec683e7180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:01.152094 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.133192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46452 (local address 127.11.60.2:43797)
0811 03:12:01.142777 (+ 9585us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:01.142784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:01.143566 (+ 782us) server_negotiation.cc:407] Connection header received
0811 03:12:01.143635 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:01.143638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:01.143695 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:01.143800 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:01.144508 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.145883 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.149579 (+ 3696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.150768 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.150893 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.151007 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:01.151548 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 03:12:01.151714 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9440,"thread_start_us":9301,"threads_started":1}
I20260811 03:12:01.152509 19630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3782773bfef54532b8db4121073fa27f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:01.153210 19469 tablet_service.cc:1467] Processing CreateTablet for tablet 3782773bfef54532b8db4121073fa27f (DEFAULT_TABLE table=test-table [id=7e869b6245e84b0c8717a0ec683e7180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:01.153692 19469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3782773bfef54532b8db4121073fa27f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:01.156666 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.144126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:39959 (local address 127.0.0.1:34442)
0811 03:12:01.147771 (+ 3645us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.147792 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:01.147954 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:01.148416 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:01.148421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:01.148446 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:01.148883 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:01.148895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.150290 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.150296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:01.154076 (+ 3780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:01.154094 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.155841 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.155848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:01.156029 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.156063 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 03:12:01.156162 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 03:12:01.156298 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3521,"thread_start_us":3429,"threads_started":1}
I20260811 03:12:01.156751 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.143501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34442 (local address 127.11.60.1:39959)
0811 03:12:01.147523 (+ 4022us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:01.147539 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:01.148058 (+ 519us) server_negotiation.cc:407] Connection header received
0811 03:12:01.148136 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:01.148141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:01.148217 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:01.148368 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:01.149045 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.150155 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.154271 (+ 4116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.155609 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.155743 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.155863 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:01.156222 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 03:12:01.156388 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3868,"thread_start_us":3732,"threads_started":1}
I20260811 03:12:01.157821 19311 tablet_service.cc:1467] Processing CreateTablet for tablet 3782773bfef54532b8db4121073fa27f (DEFAULT_TABLE table=test-table [id=7e869b6245e84b0c8717a0ec683e7180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:01.158283 19311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3782773bfef54532b8db4121073fa27f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:01.159564 19818 tablet_bootstrap.cc:492] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706: Bootstrap starting.
I20260811 03:12:01.162215 19818 tablet_bootstrap.cc:654] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:01.168964 19820 tablet_bootstrap.cc:492] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342: Bootstrap starting.
I20260811 03:12:01.171602 19820 tablet_bootstrap.cc:654] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:01.173851 19822 tablet_bootstrap.cc:492] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2: Bootstrap starting.
I20260811 03:12:01.175393 19818 tablet_bootstrap.cc:492] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706: No bootstrap required, opened a new log
I20260811 03:12:01.175583 19818 ts_tablet_manager.cc:1397] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.005s
I20260811 03:12:01.175649 19822 tablet_bootstrap.cc:654] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:01.175664 19820 tablet_bootstrap.cc:492] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342: No bootstrap required, opened a new log
I20260811 03:12:01.175835 19820 ts_tablet_manager.cc:1397] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 03:12:01.176491 19820 raft_consensus.cc:348] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } } peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } }
I20260811 03:12:01.176736 19820 raft_consensus.cc:374] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:01.176542 19818 raft_consensus.cc:348] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } } peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } }
I20260811 03:12:01.176853 19818 raft_consensus.cc:374] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:01.176877 19820 raft_consensus.cc:729] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8e84046be747fc98b50003173b3342, State: Initialized, Role: FOLLOWER
I20260811 03:12:01.176981 19818 raft_consensus.cc:729] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884a718d1b9b4654a00529bffe0ba706, State: Initialized, Role: FOLLOWER
I20260811 03:12:01.177160 19820 consensus_queue.cc:260] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } } peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } }
I20260811 03:12:01.177369 19818 consensus_queue.cc:260] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } } peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } }
I20260811 03:12:01.179036 19818 ts_tablet_manager.cc:1428] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260811 03:12:01.180298 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.179911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.3:51905)
0811 03:12:01.180085 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.180177 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 03:12:01.183656 19700 heartbeater.cc:503] Master 127.11.60.61:42747 was elected leader, sending a full tablet report...
I20260811 03:12:01.184242 19820 ts_tablet_manager.cc:1428] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260811 03:12:01.184469 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.183730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.3:60683)
0811 03:12:01.184187 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.184279 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338}
I20260811 03:12:01.185230 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.184805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.2:35779)
0811 03:12:01.184958 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.185078 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 03:12:01.186390 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.185928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.2:47477)
0811 03:12:01.186125 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.186214 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 03:12:01.188318 19822 tablet_bootstrap.cc:492] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2: No bootstrap required, opened a new log
I20260811 03:12:01.188457 19822 ts_tablet_manager.cc:1397] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.003s
I20260811 03:12:01.189105 19822 raft_consensus.cc:348] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } } peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } }
I20260811 03:12:01.189299 19822 raft_consensus.cc:374] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:01.189368 19822 raft_consensus.cc:729] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cfcb8537b49453096cea3274246b3d2, State: Initialized, Role: FOLLOWER
I20260811 03:12:01.189600 19822 consensus_queue.cc:260] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } } peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } }
I20260811 03:12:01.190423 19822 ts_tablet_manager.cc:1428] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 03:12:01.191190 19799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.190761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.1:46229)
0811 03:12:01.190974 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.191057 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 03:12:01.192329 19799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.191872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.1:36723)
0811 03:12:01.192078 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.192152 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 03:12:01.192622 19382 heartbeater.cc:503] Master 127.11.60.61:42747 was elected leader, sending a full tablet report...
I20260811 03:12:01.306980 19826 raft_consensus.cc:482] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:12:01.307299 19826 raft_consensus.cc:504] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } } peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } }
I20260811 03:12:01.309262 19826 leader_election.cc:288] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884a718d1b9b4654a00529bffe0ba706 (127.11.60.3:42001), 9cfcb8537b49453096cea3274246b3d2 (127.11.60.1:39959)
I20260811 03:12:01.316258 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.309801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:50249 (local address 127.11.60.3:42001)
0811 03:12:01.310053 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:01.310059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:01.310184 (+ 125us) server_negotiation.cc:407] Connection header received
0811 03:12:01.310418 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:01.310426 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:01.310519 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:01.310612 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:01.311109 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.312098 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.314157 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.315240 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.315344 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.315428 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:01.315849 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 03:12:01.315945 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 03:12:01.316287 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.309722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:42001 (local address 127.11.60.2:50249)
0811 03:12:01.310062 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.310078 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:01.310246 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:01.310667 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:01.310672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:01.310708 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:01.310978 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:01.310987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.312237 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.312243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:01.314011 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:01.314023 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.315463 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.315471 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:01.315661 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.315704 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 03:12:01.315817 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 03:12:01.315963 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":3}
I20260811 03:12:01.316996 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.310262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:47275 (local address 127.11.60.1:39959)
0811 03:12:01.310564 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:01.310572 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:01.310974 (+ 402us) server_negotiation.cc:407] Connection header received
0811 03:12:01.311185 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:01.311189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:01.311247 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:01.311317 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:01.311893 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.312882 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.314853 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.315933 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.316061 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.316165 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:01.316607 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 03:12:01.316749 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 03:12:01.317219 19650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3782773bfef54532b8db4121073fa27f" candidate_uuid: "ee8e84046be747fc98b50003173b3342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884a718d1b9b4654a00529bffe0ba706" is_pre_election: true
I20260811 03:12:01.317502 19829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.309960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:39959 (local address 127.11.60.2:47275)
0811 03:12:01.310863 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.310886 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:01.311025 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:01.311362 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:01.311366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:01.311390 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:01.311740 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:01.311752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.313017 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.313020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:01.314700 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:01.314725 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.316163 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.316169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:01.316377 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.316417 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 03:12:01.317178 (+ 761us) client_negotiation.cc:240] Negotiation successful
0811 03:12:01.317282 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":533,"threads_started":1}
I20260811 03:12:01.317699 19650 raft_consensus.cc:2454] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee8e84046be747fc98b50003173b3342 in term 0.
I20260811 03:12:01.318254 19333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3782773bfef54532b8db4121073fa27f" candidate_uuid: "ee8e84046be747fc98b50003173b3342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cfcb8537b49453096cea3274246b3d2" is_pre_election: true
I20260811 03:12:01.318333 19421 leader_election.cc:302] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 884a718d1b9b4654a00529bffe0ba706, ee8e84046be747fc98b50003173b3342; no voters:
I20260811 03:12:01.318536 19333 raft_consensus.cc:2454] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee8e84046be747fc98b50003173b3342 in term 0.
I20260811 03:12:01.318819 19826 raft_consensus.cc:2781] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:12:01.318961 19826 raft_consensus.cc:482] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:12:01.319079 19826 raft_consensus.cc:3037] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:01.321213 19826 raft_consensus.cc:504] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } } peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } }
I20260811 03:12:01.321780 19826 leader_election.cc:288] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [CANDIDATE]: Term 1 election: Requested vote from peers 884a718d1b9b4654a00529bffe0ba706 (127.11.60.3:42001), 9cfcb8537b49453096cea3274246b3d2 (127.11.60.1:39959)
I20260811 03:12:01.322345 19333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3782773bfef54532b8db4121073fa27f" candidate_uuid: "ee8e84046be747fc98b50003173b3342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cfcb8537b49453096cea3274246b3d2"
I20260811 03:12:01.322616 19333 raft_consensus.cc:3037] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:01.322731 19650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3782773bfef54532b8db4121073fa27f" candidate_uuid: "ee8e84046be747fc98b50003173b3342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884a718d1b9b4654a00529bffe0ba706"
I20260811 03:12:01.322937 19650 raft_consensus.cc:3037] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:01.324738 19333 raft_consensus.cc:2454] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee8e84046be747fc98b50003173b3342 in term 1.
I20260811 03:12:01.325073 19650 raft_consensus.cc:2454] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee8e84046be747fc98b50003173b3342 in term 1.
I20260811 03:12:01.325145 19423 leader_election.cc:302] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cfcb8537b49453096cea3274246b3d2, ee8e84046be747fc98b50003173b3342; no voters:
I20260811 03:12:01.325546 19826 raft_consensus.cc:2781] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:01.326018 19826 raft_consensus.cc:686] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 1 LEADER]: Becoming Leader. State: Replica: ee8e84046be747fc98b50003173b3342, State: Running, Role: LEADER
I20260811 03:12:01.326561 19548 heartbeater.cc:503] Master 127.11.60.61:42747 was elected leader, sending a full tablet report...
I20260811 03:12:01.326473 19826 consensus_queue.cc:237] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } } peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } }
I20260811 03:12:01.327592 19829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.327146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.2:46099)
0811 03:12:01.327307 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.327450 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 03:12:01.330376 18984 catalog_manager.cc:5495] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee8e84046be747fc98b50003173b3342 (127.11.60.2). New cstate: current_term: 1 leader_uuid: "ee8e84046be747fc98b50003173b3342" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee8e84046be747fc98b50003173b3342" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 43797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 } health_report { overall_health: UNKNOWN } } }
I20260811 03:12:01.380168 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.375051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46458 (local address 127.11.60.2:43797)
0811 03:12:01.375227 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:01.375232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:01.375250 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:12:01.375418 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:01.375422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:01.375509 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:01.375648 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:12:01.376040 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.376748 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.378131 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.378606 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.378701 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.379109 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:12:01.379657 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:12:01.379764 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:01.379877 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 03:12:01.379945 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 03:12:01.384466 19333 raft_consensus.cc:1261] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2 [term 1 FOLLOWER]: Refusing update from remote peer ee8e84046be747fc98b50003173b3342: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:01.384904 19650 raft_consensus.cc:1261] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706 [term 1 FOLLOWER]: Refusing update from remote peer ee8e84046be747fc98b50003173b3342: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:01.385273 19830 consensus_queue.cc:1035] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cfcb8537b49453096cea3274246b3d2" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 39959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:01.385730 19826 consensus_queue.cc:1035] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [LEADER]: Connected to new peer: Peer: permanent_uuid: "884a718d1b9b4654a00529bffe0ba706" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:01.388504 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.387956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.3:46601)
0811 03:12:01.388174 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.388277 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 03:12:01.388574 19799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.387934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.1:37547)
0811 03:12:01.388222 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.388340 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260811 03:12:01.393980 19833 mvcc.cc:205] Tried to move back new op lower bound from 7317167805983494144 to 7317167805757165568. Current Snapshot: MvccSnapshot[applied={T|T < 7317167805983494144}]
I20260811 03:12:01.395750 19829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.395185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.2:54177)
0811 03:12:01.395454 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.395558 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 03:12:01.395926 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.395355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.3:46441)
0811 03:12:01.395639 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.395743 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260811 03:12:01.398962 19835 mvcc.cc:205] Tried to move back new op lower bound from 7317167805983494144 to 7317167805757165568. Current Snapshot: MvccSnapshot[applied={T|T < 7317167805983494144 or (T in {7317167805983494144})}]
I20260811 03:12:01.404060 19799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.403321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.1:58337)
0811 03:12:01.403693 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.403824 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260811 03:12:01.404881 19829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.403021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.11.60.2:36771)
0811 03:12:01.404499 (+ 1478us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.404621 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1323}
I20260811 03:12:01.413209 18985 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:01.413477 18985 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:01.419576 18985 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f2b972980a94c26b2d8932f832ed983: Sending DeleteTablet for 3 replicas of tablet 3782773bfef54532b8db4121073fa27f
I20260811 03:12:01.420586 19630 tablet_service.cc:1514] Processing DeleteTablet for tablet 3782773bfef54532b8db4121073fa27f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:01 UTC) from {username='oryx', principal='oryx/127.11.60.61@KRBTEST.COM'} at 127.0.0.1:43038
I20260811 03:12:01.420754 19469 tablet_service.cc:1514] Processing DeleteTablet for tablet 3782773bfef54532b8db4121073fa27f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:01 UTC) from {username='oryx', principal='oryx/127.11.60.61@KRBTEST.COM'} at 127.0.0.1:46452
I20260811 03:12:01.421027 19311 tablet_service.cc:1514] Processing DeleteTablet for tablet 3782773bfef54532b8db4121073fa27f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:01 UTC) from {username='oryx', principal='oryx/127.11.60.61@KRBTEST.COM'} at 127.0.0.1:34442
I20260811 03:12:01.423655 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 19165
I20260811 03:12:01.437752 19844 tablet_replica.cc:330] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342: stopping tablet replica
I20260811 03:12:01.438062 19844 raft_consensus.cc:2229] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:12:01.438854 19844 raft_consensus.cc:2258] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:01.442809 19844 ts_tablet_manager.cc:1905] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:01.450083 19845 tablet_replica.cc:330] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2: stopping tablet replica
I20260811 03:12:01.450403 19845 raft_consensus.cc:2229] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:01.450718 19845 raft_consensus.cc:2258] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:01.451224 19844 ts_tablet_manager.cc:1918] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:01.451381 19843 tablet_replica.cc:330] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706: stopping tablet replica
I20260811 03:12:01.451452 19844 log.cc:1196] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/3782773bfef54532b8db4121073fa27f
I20260811 03:12:01.451587 19843 raft_consensus.cc:2229] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:01.451900 19843 raft_consensus.cc:2258] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:01.452191 19844 ts_tablet_manager.cc:1939] T 3782773bfef54532b8db4121073fa27f P ee8e84046be747fc98b50003173b3342: Deleting consensus metadata
I20260811 03:12:01.453572 19843 ts_tablet_manager.cc:1905] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:01.453779 18970 catalog_manager.cc:4841] TS ee8e84046be747fc98b50003173b3342 (127.11.60.2:43797): tablet 3782773bfef54532b8db4121073fa27f (table test-table [id=7e869b6245e84b0c8717a0ec683e7180]) successfully deleted
I20260811 03:12:01.454141 19845 ts_tablet_manager.cc:1905] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:01.460183 19843 ts_tablet_manager.cc:1918] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:01.460379 19843 log.cc:1196] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/3782773bfef54532b8db4121073fa27f
I20260811 03:12:01.460942 19845 ts_tablet_manager.cc:1918] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:01.461128 19845 log.cc:1196] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/3782773bfef54532b8db4121073fa27f
I20260811 03:12:01.461649 19845 ts_tablet_manager.cc:1939] T 3782773bfef54532b8db4121073fa27f P 9cfcb8537b49453096cea3274246b3d2: Deleting consensus metadata
I20260811 03:12:01.462968 18970 catalog_manager.cc:4841] TS 9cfcb8537b49453096cea3274246b3d2 (127.11.60.1:39959): tablet 3782773bfef54532b8db4121073fa27f (table test-table [id=7e869b6245e84b0c8717a0ec683e7180]) successfully deleted
I20260811 03:12:01.464295 19843 ts_tablet_manager.cc:1939] T 3782773bfef54532b8db4121073fa27f P 884a718d1b9b4654a00529bffe0ba706: Deleting consensus metadata
I20260811 03:12:01.465771 18969 catalog_manager.cc:4841] TS 884a718d1b9b4654a00529bffe0ba706 (127.11.60.3:42001): tablet 3782773bfef54532b8db4121073fa27f (table test-table [id=7e869b6245e84b0c8717a0ec683e7180]) successfully deleted
I20260811 03:12:01.469722 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.425880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.11.60.1:39959)
0811 03:12:01.426115 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:01.426125 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:01.445957 (+ 19832us) server_negotiation.cc:407] Connection header received
0811 03:12:01.446588 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:01.446597 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:01.446713 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:01.446808 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:01.452291 (+ 5483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.453481 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.456345 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.457091 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.457210 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.461580 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:01.461603 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:01.461628 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:01.461692 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:01.466070 (+ 4378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:01.467123 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:01.467133 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:01.467137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:01.467302 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:01.467769 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:01.467774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:01.467777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:01.468287 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:01.468624 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:01.468980 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 03:12:01.469203 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 03:12:01.470351 19374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:01.497961 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.497407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35682)
0811 03:12:01.497643 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.497768 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 03:12:01.499262 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.498785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35696)
0811 03:12:01.498965 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.499079 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 03:12:01.576269 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 19165
I20260811 03:12:01.605619 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 19395
I20260811 03:12:01.625450 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.606639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46474 (local address 127.11.60.2:43797)
0811 03:12:01.606829 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:01.606838 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:01.606906 (+ 68us) server_negotiation.cc:407] Connection header received
0811 03:12:01.607117 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:01.607123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:01.607221 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:01.607327 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:01.609156 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.610784 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.612742 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.613505 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.613633 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.617360 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:01.617380 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:01.617401 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:01.617443 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:01.621769 (+ 4326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:01.622645 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:01.622655 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:01.622658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:01.622865 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:01.623343 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:01.623349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:01.623352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:01.623859 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:01.624189 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:01.624774 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 03:12:01.625015 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 03:12:01.626293 19539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:01.750037 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 19395
I20260811 03:12:01.772876 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 19564
I20260811 03:12:01.787727 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.773695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43040 (local address 127.11.60.3:42001)
0811 03:12:01.773904 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:01.773910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:01.773930 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:12:01.774105 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:01.774110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:01.774190 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:01.774298 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:01.775755 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.776519 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.778100 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:01.778603 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:01.778747 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:01.781464 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:01.781489 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:01.781507 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:01.781553 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:01.784693 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:01.785434 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:01.785448 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:01.785451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:01.785602 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:01.786000 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:01.786006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:01.786009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:01.786456 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:01.786792 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:01.787119 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 03:12:01.787358 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 03:12:01.788583 19692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:01.913487 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 19564
I20260811 03:12:01.925889 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.925331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35706)
0811 03:12:01.925548 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.925673 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 03:12:01.927122 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:01.926658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42751 (local address 127.0.0.1:35722)
0811 03:12:01.926840 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:01.926942 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 03:12:01.941144 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 18834
I20260811 03:12:01.942196 19018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:02.092043 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 18834
W20260811 03:12:02.117684 19177 connection.cc:535] server connection from 127.0.0.1:53620 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 03:12:02.118047 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 18945
I20260811 03:12:02.119306 19220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:02.239595 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 18945
2026-08-11T03:12:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12279 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:12:02 dist-test-slave-tm1g krb5kdc[19855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:12:02 dist-test-slave-tm1g krb5kdc[19855](info): set up 2 sockets
Aug 11 03:12:02 dist-test-slave-tm1g krb5kdc[19855](info): commencing operation
krb5kdc: starting...
W20260811 03:12:03.449575 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:12:03 dist-test-slave-tm1g krb5kdc[19855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:12:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:12:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.62.keytab.
Entry for principal oryx/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.62.keytab.
I20260811 03:12:03.735275 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40553
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.62.keytab
--principal=oryx/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:40553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:04.086910 19870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:04.087201 19870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:04.087381 19870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:04.094661 19870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:12:04.094772 19870 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 03:12:04.094853 19870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:04.094946 19870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:04.095012 19870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:12:04.095100 19870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:12:04.105340 19870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:40553
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.62.keytab
--principal=oryx/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:04.106746 19870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:04.108126 19870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:04.120748 19876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:04.120800 19877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:04.120702 19879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:04.121541 19870 server_base.cc:1034] running on GCE node
I20260811 03:12:04.122409 19870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:04.123880 19870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:04.125145 19870 hybrid_clock.cc:648] HybridClock initialized: now 1786417924125088 us; error 36 us; skew 500 ppm
I20260811 03:12:04.125579 19870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:04 dist-test-slave-tm1g krb5kdc[19855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417924, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:04.132079 19870 init.cc:377] Logged in from keytab as oryx/127.11.60.62@KRBTEST.COM (short username oryx)
I20260811 03:12:04.133671 19870 webserver.cc:460] Webserver started at http://127.11.60.62:35361/ using document root <none> and password file <none>
I20260811 03:12:04.134500 19870 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:04.134599 19870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:04.134891 19870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:04.138134 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "59aa121f37364f5b8905751cfec5e7ed"
format_stamp: "Formatted at 2026-08-11 03:12:04 on dist-test-slave-tm1g"
server_key: "3ae2002ae2be33b109e0772a3d9229dd"
server_key_iv: "f617ffb14e591cd0b375be7dd4a8d5f0"
server_key_version: "encryptionkey@0"
I20260811 03:12:04.139002 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "59aa121f37364f5b8905751cfec5e7ed"
format_stamp: "Formatted at 2026-08-11 03:12:04 on dist-test-slave-tm1g"
server_key: "3ae2002ae2be33b109e0772a3d9229dd"
server_key_iv: "f617ffb14e591cd0b375be7dd4a8d5f0"
server_key_version: "encryptionkey@0"
I20260811 03:12:04.144425 19870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:12:04.148243 19886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:04.149724 19870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:12:04.149904 19870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "59aa121f37364f5b8905751cfec5e7ed"
format_stamp: "Formatted at 2026-08-11 03:12:04 on dist-test-slave-tm1g"
server_key: "3ae2002ae2be33b109e0772a3d9229dd"
server_key_iv: "f617ffb14e591cd0b375be7dd4a8d5f0"
server_key_version: "encryptionkey@0"
I20260811 03:12:04.150113 19870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:04.164122 19870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:04.168627 19870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:04.168915 19870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:04.192178 19870 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:40553
I20260811 03:12:04.192217 19951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:40553 every 8 connection(s)
I20260811 03:12:04.193658 19870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:12:04.196969 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 19870
I20260811 03:12:04.197207 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:12:04.197901 11504 external_mini_cluster.cc:1428] Setting key 10c82a00c894199b23ca5d0017b803f7
I20260811 03:12:04.197944 19952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:04.212966 19952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed: Bootstrap starting.
Aug 11 03:12:04 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:12:04.217281 19952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:04.218776 19952 log.cc:824] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:04.222551 19952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed: No bootstrap required, opened a new log
I20260811 03:12:04.223877 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:04.200022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.11.60.62:40553)
0811 03:12:04.200876 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:04.200892 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:04.200948 (+ 56us) server_negotiation.cc:407] Connection header received
0811 03:12:04.203184 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:04.203219 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:04.203759 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:04.204223 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:04.205569 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:04.207671 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:04.209053 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:04.210178 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:04.210425 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:04.214490 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:04.214517 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:04.214539 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:04.214609 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:04.218974 (+ 4365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:04.219898 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:04.219905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:04.219911 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:04.220094 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:04.220502 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:04.220507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:04.220510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:04.221229 (+ 719us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:04.221538 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:04.221955 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 03:12:04.222440 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":341,"threads_started":1}
I20260811 03:12:04.231355 19952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59aa121f37364f5b8905751cfec5e7ed" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40553 } }
I20260811 03:12:04.231775 19952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:04.231855 19952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa121f37364f5b8905751cfec5e7ed, State: Initialized, Role: FOLLOWER
I20260811 03:12:04.232425 19952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59aa121f37364f5b8905751cfec5e7ed" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40553 } }
I20260811 03:12:04.232640 19952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:12:04.232745 19952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:12:04.232869 19952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:04.235086 19952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59aa121f37364f5b8905751cfec5e7ed" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40553 } }
I20260811 03:12:04.235562 19952 leader_election.cc:302] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59aa121f37364f5b8905751cfec5e7ed; no voters:
I20260811 03:12:04.236188 19952 leader_election.cc:288] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:12:04.236361 19957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:04.237375 19957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [term 1 LEADER]: Becoming Leader. State: Replica: 59aa121f37364f5b8905751cfec5e7ed, State: Running, Role: LEADER
I20260811 03:12:04.237954 19957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59aa121f37364f5b8905751cfec5e7ed" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40553 } }
I20260811 03:12:04.238739 19952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:12:04.240865 19958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59aa121f37364f5b8905751cfec5e7ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59aa121f37364f5b8905751cfec5e7ed" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40553 } } }
I20260811 03:12:04.240964 19959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59aa121f37364f5b8905751cfec5e7ed. Latest consensus state: current_term: 1 leader_uuid: "59aa121f37364f5b8905751cfec5e7ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59aa121f37364f5b8905751cfec5e7ed" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40553 } } }
I20260811 03:12:04.241258 19958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:04.241369 19959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:04.242331 19963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:12:04.247902 19963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:12:04.257714 19963 catalog_manager.cc:1347] Generated new cluster ID: be53eee5d006471c8d24201d942a50d6
I20260811 03:12:04.257884 19963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:12:04.274865 19963 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:12:04.276233 19963 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:12:04.288517 19963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed: Generated new TSK 0
I20260811 03:12:04.289618 19963 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab.
Entry for principal oryx/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab.
I20260811 03:12:04.377815 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:40553
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab
--principal=oryx/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:04.726385 19979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:04.726691 19979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:04.726877 19979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:04.734429 19979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:04.734555 19979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:04.734716 19979 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 03:12:04.734829 19979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:12:04.745307 19979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab
--principal=oryx/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.60.62:40553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:04.746786 19979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:04.748299 19979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:04.761950 19984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:04.761958 19985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:04.762629 19987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:04.763078 19979 server_base.cc:1034] running on GCE node
I20260811 03:12:04.763739 19979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:04.764643 19979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:04.765882 19979 hybrid_clock.cc:648] HybridClock initialized: now 1786417924765808 us; error 38 us; skew 500 ppm
I20260811 03:12:04.766208 19979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:04 dist-test-slave-tm1g krb5kdc[19855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417924, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:04.769630 19979 init.cc:377] Logged in from keytab as oryx/127.11.60.1@KRBTEST.COM (short username oryx)
I20260811 03:12:04.771209 19979 webserver.cc:460] Webserver started at http://127.11.60.1:44139/ using document root <none> and password file <none>
I20260811 03:12:04.771986 19979 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:04.772081 19979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:04.772439 19979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:04.775743 19979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "28207ee872d94820bd3be90d5b2cee7b"
format_stamp: "Formatted at 2026-08-11 03:12:04 on dist-test-slave-tm1g"
server_key: "b61b7e0743187534fbf8064a541eb218"
server_key_iv: "fa78f2a510adfec4aa54fe5362852e2a"
server_key_version: "encryptionkey@0"
I20260811 03:12:04.776628 19979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "28207ee872d94820bd3be90d5b2cee7b"
format_stamp: "Formatted at 2026-08-11 03:12:04 on dist-test-slave-tm1g"
server_key: "b61b7e0743187534fbf8064a541eb218"
server_key_iv: "fa78f2a510adfec4aa54fe5362852e2a"
server_key_version: "encryptionkey@0"
I20260811 03:12:04.782115 19979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:12:04.785931 19994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:04.787657 19979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:12:04.787859 19979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "28207ee872d94820bd3be90d5b2cee7b"
format_stamp: "Formatted at 2026-08-11 03:12:04 on dist-test-slave-tm1g"
server_key: "b61b7e0743187534fbf8064a541eb218"
server_key_iv: "fa78f2a510adfec4aa54fe5362852e2a"
server_key_version: "encryptionkey@0"
I20260811 03:12:04.788100 19979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:04.810884 19979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:04.815740 19979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:04.816066 19979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:04.819010 19979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:04.819134 19979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:04.819314 19979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:04.819406 19979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 03:12:04 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417924, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:12:04.845230 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:04.819813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:41077 (local address 127.11.60.62:40553)
0811 03:12:04.820368 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:04.820382 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:04.821636 (+ 1254us) server_negotiation.cc:407] Connection header received
0811 03:12:04.824272 (+ 2636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:04.824277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:04.824389 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:04.824517 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:04.826982 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:04.828051 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:04.830839 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:04.831263 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:04.831420 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:04.837200 (+ 5780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:04.837206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:04.837209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:04.837246 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:04.841567 (+ 4321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:04.842730 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:04.842734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:04.842736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:04.842860 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:04.843336 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:04.843340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:04.843342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:04.843626 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:04.843799 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:04.844587 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 03:12:04.844832 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":287,"threads_started":1}
I20260811 03:12:04.846529 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:04.820483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40553 (local address 127.11.60.1:41077)
0811 03:12:04.821434 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:04.821488 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:04.823854 (+ 2366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:04.824782 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:04.824791 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:04.825604 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:04.826692 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:04.826730 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:04.828215 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:04.828220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:04.830663 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:04.830680 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:04.831547 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:04.831556 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:04.831830 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:04.832796 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:04.832845 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:04.836919 (+ 4074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:04.841773 (+ 4854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:04.841793 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:04.841830 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:04.842547 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:04.842987 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:04.842992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:04.842995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:04.843205 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:04.843759 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:04.843770 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:04.844169 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 03:12:04.844449 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 03:12:04.845027 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":428,"threads_started":1}
I20260811 03:12:04.870913 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:04.854731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:58741 (local address 127.11.60.62:40553)
0811 03:12:04.854933 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:04.854939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:04.855880 (+ 941us) server_negotiation.cc:407] Connection header received
0811 03:12:04.856092 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:04.856097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:04.856170 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:04.856271 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:04.857761 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:04.858922 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:04.860855 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:04.861294 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:04.861435 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:04.863863 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:04.863868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:04.863872 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:04.863903 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:04.867347 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:04.868307 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:04.868315 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:04.868321 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:04.868520 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:04.869069 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:04.869073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:04.869075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:04.869368 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:04.869483 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:04.870142 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 03:12:04.870510 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 03:12:04.871394 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:04.854791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40553 (local address 127.11.60.1:58741)
0811 03:12:04.855733 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:04.855769 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:04.855951 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:04.856303 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:04.856308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:04.856986 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:04.857612 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:04.857625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:04.859063 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:04.859068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:04.860698 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:04.860715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:04.861519 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:04.861529 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:04.861699 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:04.862388 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:04.862415 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:04.863606 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:04.867515 (+ 3909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:04.867519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:04.867522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:04.868184 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:04.868668 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:04.868676 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:04.868683 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:04.868934 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:04.869498 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:04.869506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:04.869778 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 03:12:04.870017 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 03:12:04.870659 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":560,"threads_started":1}
I20260811 03:12:04.871528 19979 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:35415
I20260811 03:12:04.871533 20124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:35415 every 8 connection(s)
I20260811 03:12:04.873332 19979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:12:04.876073 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 19979
I20260811 03:12:04.876224 20125 heartbeater.cc:347] Connected to a master server at 127.11.60.62:40553
I20260811 03:12:04.876305 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:12:04.876551 20125 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:04.877213 11504 external_mini_cluster.cc:1428] Setting key 9c31542d69325f1ed1d22c607e349832
I20260811 03:12:04.877444 20125 heartbeater.cc:511] Master 127.11.60.62:40553 requested a full tablet report, sending...
I20260811 03:12:04.879980 19904 ts_manager.cc:194] Registered new tserver with Master: 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415)
I20260811 03:12:04.881490 19904 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:41077
WARNING: no policy specified for oryx/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab.
Entry for principal oryx/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab.
I20260811 03:12:04.938159 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:40553
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab
--principal=oryx/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:05.286609 20129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:05.286938 20129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:05.287104 20129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:05.294870 20129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:05.295022 20129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:05.295156 20129 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 03:12:05.295285 20129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:12:05.305420 20129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab
--principal=oryx/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.60.62:40553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:05.306834 20129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:05.308218 20129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:05.322201 20135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:05.322214 20138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:05.322275 20136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:05.323045 20129 server_base.cc:1034] running on GCE node
I20260811 03:12:05.323768 20129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:05.324797 20129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:05.326077 20129 hybrid_clock.cc:648] HybridClock initialized: now 1786417925326003 us; error 40 us; skew 500 ppm
I20260811 03:12:05.326485 20129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:05 dist-test-slave-tm1g krb5kdc[19855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417925, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:05.329975 20129 init.cc:377] Logged in from keytab as oryx/127.11.60.2@KRBTEST.COM (short username oryx)
I20260811 03:12:05.331653 20129 webserver.cc:460] Webserver started at http://127.11.60.2:42897/ using document root <none> and password file <none>
I20260811 03:12:05.332412 20129 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:05.332518 20129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:05.332805 20129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:05.335947 20129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc"
format_stamp: "Formatted at 2026-08-11 03:12:05 on dist-test-slave-tm1g"
server_key: "7464cf556c892cf42b1a0520a9d3ad96"
server_key_iv: "20feb02f2576b9905f2795bb0ed72e8e"
server_key_version: "encryptionkey@0"
I20260811 03:12:05.336740 20129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc"
format_stamp: "Formatted at 2026-08-11 03:12:05 on dist-test-slave-tm1g"
server_key: "7464cf556c892cf42b1a0520a9d3ad96"
server_key_iv: "20feb02f2576b9905f2795bb0ed72e8e"
server_key_version: "encryptionkey@0"
I20260811 03:12:05.342448 20129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 03:12:05.346002 20145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:05.347625 20129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 03:12:05.347797 20129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc"
format_stamp: "Formatted at 2026-08-11 03:12:05 on dist-test-slave-tm1g"
server_key: "7464cf556c892cf42b1a0520a9d3ad96"
server_key_iv: "20feb02f2576b9905f2795bb0ed72e8e"
server_key_version: "encryptionkey@0"
I20260811 03:12:05.348062 20129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:05.364339 20129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:05.368773 20129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:05.369061 20129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:05.372022 20129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:05.372184 20129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 03:12:05.372372 20129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:05.372498 20129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 03:12:05 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417925, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:12:05.398718 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:05.373172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:34821 (local address 127.11.60.62:40553)
0811 03:12:05.373668 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:05.373677 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:05.375707 (+ 2030us) server_negotiation.cc:407] Connection header received
0811 03:12:05.378271 (+ 2564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:05.378277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:05.378395 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:05.378524 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:05.381181 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.382360 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.384787 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.385688 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.385855 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:05.390622 (+ 4767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:05.390634 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:05.390637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:05.390696 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:05.394363 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:05.395222 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:05.395230 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:05.395235 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:05.395419 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:05.396064 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:05.396068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:05.396071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:05.396426 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:05.396589 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:05.397867 (+ 1278us) server_negotiation.cc:299] Negotiation successful
0811 03:12:05.398184 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":251,"threads_started":1}
I20260811 03:12:05.400727 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:05.374470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40553 (local address 127.11.60.2:34821)
0811 03:12:05.375513 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:05.375572 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:05.377936 (+ 2364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:05.378844 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:05.378861 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:05.379959 (+ 1098us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:05.380895 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:05.380930 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.382542 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.382549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:05.384608 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:05.384629 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.385973 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.385986 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:05.386293 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:05.387266 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:05.387319 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:05.390361 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:05.394530 (+ 4169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:05.394539 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:05.394556 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:05.395102 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:05.395575 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:05.395581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:05.395589 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:05.395835 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:05.396624 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:05.396636 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:05.397273 (+ 637us) client_negotiation.cc:769] Sending connection context
0811 03:12:05.397748 (+ 475us) client_negotiation.cc:240] Negotiation successful
0811 03:12:05.398517 (+ 769us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":436,"threads_started":1}
I20260811 03:12:05.418220 20129 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:34567
I20260811 03:12:05.418247 20274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:34567 every 8 connection(s)
I20260811 03:12:05.419659 20129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:12:05.422219 20275 heartbeater.cc:347] Connected to a master server at 127.11.60.62:40553
I20260811 03:12:05.422519 20275 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:05.423398 20275 heartbeater.cc:511] Master 127.11.60.62:40553 requested a full tablet report, sending...
I20260811 03:12:05.425001 19904 ts_manager.cc:194] Registered new tserver with Master: f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567)
I20260811 03:12:05.425925 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 20129
I20260811 03:12:05.426030 19904 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:34821
I20260811 03:12:05.426108 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:12:05.426734 11504 external_mini_cluster.cc:1428] Setting key 5e4ee57f46a306de01302f0a83f987bc
I20260811 03:12:05.436436 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:05.407555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:51161 (local address 127.11.60.62:40553)
0811 03:12:05.407805 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:05.407812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:05.422658 (+ 14846us) server_negotiation.cc:407] Connection header received
0811 03:12:05.422894 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:05.422899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:05.422971 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:05.423105 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:05.424453 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.425558 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.427521 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.428100 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.428257 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:05.430633 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:05.430638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:05.430642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:05.430674 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:05.433422 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:05.434357 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:05.434363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:05.434367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:05.434514 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:05.434973 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:05.434978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:05.434980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:05.435348 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:05.435479 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:05.435997 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 03:12:05.436185 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":70}
I20260811 03:12:05.436825 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:05.407531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40553 (local address 127.11.60.2:51161)
0811 03:12:05.422547 (+ 15016us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:05.422571 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:05.422728 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:05.423141 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:05.423145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:05.423971 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:05.424296 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:05.424310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.425735 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.425741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:05.427379 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:05.427391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.428306 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.428361 (+ 55us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:05.428524 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:05.429235 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:05.429260 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:05.430413 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:05.433568 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:05.433575 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:05.433582 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:05.434157 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:05.434631 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:05.434639 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:05.434645 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:05.434814 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:05.435479 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:05.435487 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:05.435708 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 03:12:05.435865 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 03:12:05.436244 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14838,"thread_start_us":15301,"threads_started":1}
WARNING: no policy specified for oryx/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab.
Entry for principal oryx/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab.
I20260811 03:12:05.490403 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:40553
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab
--principal=oryx/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:05.842465 20279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:05.842833 20279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:05.843008 20279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:05.850795 20279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:05.850941 20279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:05.851081 20279 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 03:12:05.851214 20279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:12:05.861610 20279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab
--principal=oryx/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.60.62:40553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:05.863065 20279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:05.864490 20279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:05.880334 20284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:05.880357 20285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:05.880532 20287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:05.881330 20279 server_base.cc:1034] running on GCE node
I20260811 03:12:05.882040 20279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:05.883260 20279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:05.884533 20279 hybrid_clock.cc:648] HybridClock initialized: now 1786417925884443 us; error 62 us; skew 500 ppm
I20260811 03:12:05.884872 20279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:12:05.885031 20125 heartbeater.cc:503] Master 127.11.60.62:40553 was elected leader, sending a full tablet report...
Aug 11 03:12:05 dist-test-slave-tm1g krb5kdc[19855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417925, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:05.888643 20279 init.cc:377] Logged in from keytab as oryx/127.11.60.3@KRBTEST.COM (short username oryx)
I20260811 03:12:05.890169 20279 webserver.cc:460] Webserver started at http://127.11.60.3:36451/ using document root <none> and password file <none>
I20260811 03:12:05.890964 20279 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:05.891062 20279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:05.891357 20279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:05.894719 20279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "4e5bc18bf1ac45118599e8390a11229d"
format_stamp: "Formatted at 2026-08-11 03:12:05 on dist-test-slave-tm1g"
server_key: "da09d5ac9ab3e4f89d4e352afb10bf09"
server_key_iv: "d8fd2da68cde837e4c67258632d7ad45"
server_key_version: "encryptionkey@0"
I20260811 03:12:05.895512 20279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "4e5bc18bf1ac45118599e8390a11229d"
format_stamp: "Formatted at 2026-08-11 03:12:05 on dist-test-slave-tm1g"
server_key: "da09d5ac9ab3e4f89d4e352afb10bf09"
server_key_iv: "d8fd2da68cde837e4c67258632d7ad45"
server_key_version: "encryptionkey@0"
I20260811 03:12:05.900838 20279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:12:05.904779 20296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:05.906189 20279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:12:05.906405 20279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "4e5bc18bf1ac45118599e8390a11229d"
format_stamp: "Formatted at 2026-08-11 03:12:05 on dist-test-slave-tm1g"
server_key: "da09d5ac9ab3e4f89d4e352afb10bf09"
server_key_iv: "d8fd2da68cde837e4c67258632d7ad45"
server_key_version: "encryptionkey@0"
I20260811 03:12:05.906616 20279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:05.930688 20279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:05.936299 20279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:05.936597 20279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:05.939908 20279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:05.940029 20279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:05.940193 20279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:05.940267 20279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 03:12:05 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417925, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:12:05.965214 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:05.940779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:45181 (local address 127.11.60.62:40553)
0811 03:12:05.941303 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:05.941309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:05.942688 (+ 1379us) server_negotiation.cc:407] Connection header received
0811 03:12:05.945913 (+ 3225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:05.945921 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:05.946024 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:05.946171 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:05.948477 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.949718 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.951851 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.952388 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.952531 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:05.957165 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:05.957171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:05.957174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:05.957208 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:05.961360 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:05.962454 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:05.962460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:05.962472 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:05.962628 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:05.963211 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:05.963216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:05.963220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:05.963639 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:05.963802 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:05.964564 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 03:12:05.964801 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":237,"threads_started":1}
I20260811 03:12:05.966279 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:05.941498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40553 (local address 127.11.60.3:45181)
0811 03:12:05.942496 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:05.942554 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:05.945488 (+ 2934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:05.946495 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:05.946503 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:05.947398 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:05.948271 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:05.948303 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.949872 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.949887 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:05.951695 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:05.951708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.952663 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.952672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:05.952904 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:05.953773 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:05.953803 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:05.956873 (+ 3070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:05.961548 (+ 4675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:05.961561 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:05.961587 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:05.962288 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:05.962801 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:05.962812 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:05.962816 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:05.963085 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:05.963780 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:05.963792 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:05.964177 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 03:12:05.964444 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 03:12:05.965011 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":415,"threads_started":1}
I20260811 03:12:05.986940 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:05.972425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:32911 (local address 127.11.60.62:40553)
0811 03:12:05.972586 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:05.972590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:05.973259 (+ 669us) server_negotiation.cc:407] Connection header received
0811 03:12:05.973506 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:05.973510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:05.973576 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:05.973676 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:05.975260 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.976318 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.977639 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.978101 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.978248 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:05.980797 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:05.980806 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:05.980812 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:05.980864 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:05.984178 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:05.984907 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:05.984914 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:05.984918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:05.985052 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:05.985424 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:05.985429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:05.985433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:05.985735 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:05.985896 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:05.986285 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 03:12:05.986563 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 03:12:05.987058 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:05.972448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40553 (local address 127.11.60.3:32911)
0811 03:12:05.973148 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:05.973164 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:05.973345 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:05.973708 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:05.973713 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:05.974500 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:05.975096 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:05.975109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.976463 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.976469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:05.977488 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:05.977500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:05.978346 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:05.978355 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:05.978578 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:05.979470 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:05.979535 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:05.980560 (+ 1025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:05.984321 (+ 3761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:05.984326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:05.984330 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:05.984786 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:05.985162 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:05.985167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:05.985171 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:05.985304 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:05.985854 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:05.985859 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:05.986047 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 03:12:05.986189 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 03:12:05.986563 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":425,"threads_started":1}
I20260811 03:12:05.998339 20279 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:37411
I20260811 03:12:05.998374 20430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:37411 every 8 connection(s)
I20260811 03:12:05.999689 20279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:12:06.002023 20432 heartbeater.cc:347] Connected to a master server at 127.11.60.62:40553
I20260811 03:12:06.002238 20432 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:06.002959 20432 heartbeater.cc:511] Master 127.11.60.62:40553 requested a full tablet report, sending...
I20260811 03:12:06.004549 19904 ts_manager.cc:194] Registered new tserver with Master: 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411)
I20260811 03:12:06.005389 19904 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:45181
I20260811 03:12:06.008930 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 20279
I20260811 03:12:06.009131 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:12:06.009608 11504 external_mini_cluster.cc:1428] Setting key f023ff86b099ced2b7641f00d13a9523
I20260811 03:12:06.013116 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 03:12:06 dist-test-slave-tm1g krb5kdc[19855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417926, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 03:12:06 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:06 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 03:12:06.073961 20307 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:06.062555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39796 (local address 127.11.60.62:40553)
0811 03:12:06.062806 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.062814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.063014 (+ 200us) server_negotiation.cc:407] Connection header received
0811 03:12:06.063239 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.063245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.063329 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.063446 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:06.064912 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.065664 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.067006 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.067447 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.067582 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.073639 (+ 6057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39796: BlockingRecv error: recv got EOF from 127.0.0.1:39796 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 03:12:06 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417926, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:12:06.091056 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.077205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39800 (local address 127.11.60.62:40553)
0811 03:12:06.077415 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.077421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.077675 (+ 254us) server_negotiation.cc:407] Connection header received
0811 03:12:06.077813 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.077817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.077899 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.077982 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:06.079211 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.080133 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.081374 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.081821 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.081919 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.085647 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:06.085652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:06.085655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:06.085688 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:06.088287 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:06.088938 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:06.088942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:06.088944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:06.089068 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:06.089458 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:06.089462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:06.089465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:06.089756 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:06.089982 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:06.090455 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 03:12:06.090695 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 03:12:06.095726 19904 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:12:06.103018 19904 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:12:06.128489 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.117008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44766 (local address 127.11.60.2:34567)
0811 03:12:06.119705 (+ 2697us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.119713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.120278 (+ 565us) server_negotiation.cc:407] Connection header received
0811 03:12:06.120396 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.120403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.120632 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.120868 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:06.121782 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.123233 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.125673 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.127171 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.127305 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.127562 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:06.127929 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 03:12:06.128102 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2555,"thread_start_us":2417,"threads_started":1}
I20260811 03:12:06.128926 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.116407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:35415 (local address 127.0.0.1:38780)
0811 03:12:06.118400 (+ 1993us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:06.118441 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:06.118623 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:06.120132 (+ 1509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:06.120142 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:06.120175 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:06.120871 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.120896 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.122920 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.122926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.125637 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.125658 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.127561 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.127569 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.127728 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.127752 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:12:06.127849 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 03:12:06.128065 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1697,"mutex_wait_us":2,"thread_start_us":1540,"threads_started":1}
I20260811 03:12:06.129596 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.116428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:34567 (local address 127.0.0.1:44766)
0811 03:12:06.120001 (+ 3573us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:06.120022 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:06.120144 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:06.121103 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:06.121107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:06.121133 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:06.121590 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.121609 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.123371 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.123379 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.125489 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.125504 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.127380 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.127387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.127658 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.127722 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 03:12:06.129007 (+ 1285us) client_negotiation.cc:240] Negotiation successful
0811 03:12:06.129173 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3446,"mutex_wait_us":28,"thread_start_us":3309,"threads_started":1}
I20260811 03:12:06.129739 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.117111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.11.60.1:35415)
0811 03:12:06.119393 (+ 2282us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.119405 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.119440 (+ 35us) server_negotiation.cc:407] Connection header received
0811 03:12:06.119549 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.119556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.119837 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.120290 (+ 453us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:06.121074 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.122751 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.125854 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.127326 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.128898 (+ 1572us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.129141 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:06.129220 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 03:12:06.129406 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2112,"thread_start_us":1978,"threads_started":1}
I20260811 03:12:06.130200 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.121145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:37411 (local address 127.0.0.1:55470)
0811 03:12:06.122148 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:06.122162 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:06.122291 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:06.122785 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:06.122790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:06.122814 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:06.123269 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.123280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.125381 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.125387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.127028 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.127040 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.129282 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.129291 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.129480 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.129512 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 03:12:06.129627 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 03:12:06.129902 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"thread_start_us":786,"threads_started":1}
I20260811 03:12:06.130705 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.120455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.11.60.3:37411)
0811 03:12:06.121677 (+ 1222us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.121685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.122256 (+ 571us) server_negotiation.cc:407] Connection header received
0811 03:12:06.122459 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.122465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.122614 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.122807 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:06.123534 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.125224 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.127181 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.128712 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.128849 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.129117 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:06.129977 (+ 860us) server_negotiation.cc:299] Negotiation successful
0811 03:12:06.130154 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1058,"thread_start_us":859,"threads_started":1}
I20260811 03:12:06.133358 20051 tablet_service.cc:1467] Processing CreateTablet for tablet cbd7d55f3811466787d2b194314c3052 (DEFAULT_TABLE table=test-table [id=106c642f39e24b3e9c99677ed36af1f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:06.133376 20200 tablet_service.cc:1467] Processing CreateTablet for tablet cbd7d55f3811466787d2b194314c3052 (DEFAULT_TABLE table=test-table [id=106c642f39e24b3e9c99677ed36af1f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:06.134023 20351 tablet_service.cc:1467] Processing CreateTablet for tablet cbd7d55f3811466787d2b194314c3052 (DEFAULT_TABLE table=test-table [id=106c642f39e24b3e9c99677ed36af1f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:06.134893 20051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbd7d55f3811466787d2b194314c3052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:06.134917 20200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbd7d55f3811466787d2b194314c3052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:06.135396 20351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbd7d55f3811466787d2b194314c3052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:06.148198 20457 tablet_bootstrap.cc:492] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: Bootstrap starting.
I20260811 03:12:06.150740 20458 tablet_bootstrap.cc:492] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Bootstrap starting.
I20260811 03:12:06.151103 20459 tablet_bootstrap.cc:492] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: Bootstrap starting.
I20260811 03:12:06.152276 20457 tablet_bootstrap.cc:654] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:06.153681 20457 log.cc:824] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:06.157398 20457 tablet_bootstrap.cc:492] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: No bootstrap required, opened a new log
I20260811 03:12:06.157723 20457 ts_tablet_manager.cc:1397] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 03:12:06.157756 20458 tablet_bootstrap.cc:654] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:06.158861 20459 tablet_bootstrap.cc:654] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:06.159134 20458 log.cc:824] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:06.160180 20459 log.cc:824] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:06.162496 20458 tablet_bootstrap.cc:492] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: No bootstrap required, opened a new log
I20260811 03:12:06.162779 20459 tablet_bootstrap.cc:492] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: No bootstrap required, opened a new log
I20260811 03:12:06.162786 20458 ts_tablet_manager.cc:1397] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260811 03:12:06.163039 20459 ts_tablet_manager.cc:1397] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260811 03:12:06.168680 20457 raft_consensus.cc:348] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.169193 20457 raft_consensus.cc:374] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:06.169299 20457 raft_consensus.cc:729] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28207ee872d94820bd3be90d5b2cee7b, State: Initialized, Role: FOLLOWER
I20260811 03:12:06.169972 20457 consensus_queue.cc:260] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.171850 20457 ts_tablet_manager.cc:1428] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260811 03:12:06.171607 20459 raft_consensus.cc:348] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.172040 20459 raft_consensus.cc:374] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:06.172133 20459 raft_consensus.cc:729] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5bc18bf1ac45118599e8390a11229d, State: Initialized, Role: FOLLOWER
I20260811 03:12:06.172713 20459 consensus_queue.cc:260] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.173904 20432 heartbeater.cc:503] Master 127.11.60.62:40553 was elected leader, sending a full tablet report...
I20260811 03:12:06.174533 20459 ts_tablet_manager.cc:1428] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 03:12:06.173628 20458 raft_consensus.cc:348] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.175612 20458 raft_consensus.cc:374] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:06.175756 20458 raft_consensus.cc:729] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fba21ffb2f4dc68f88dbfe7d5d76cc, State: Initialized, Role: FOLLOWER
I20260811 03:12:06.176472 20458 consensus_queue.cc:260] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.176839 20463 raft_consensus.cc:482] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:12:06.177126 20463 raft_consensus.cc:504] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.179416 20463 leader_election.cc:288] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567), 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411)
I20260811 03:12:06.180380 20275 heartbeater.cc:503] Master 127.11.60.62:40553 was elected leader, sending a full tablet report...
I20260811 03:12:06.181375 20458 ts_tablet_manager.cc:1428] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Time spent starting tablet: real 0.018s user 0.013s sys 0.002s
I20260811 03:12:06.188647 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.180209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:39667 (local address 127.11.60.2:34567)
0811 03:12:06.180418 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.180425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.182191 (+ 1766us) server_negotiation.cc:407] Connection header received
0811 03:12:06.182493 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.182501 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.182582 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.182651 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:06.183545 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.184583 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.186887 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.187669 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.187826 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.187940 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:06.188226 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 03:12:06.188356 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 03:12:06.188899 20467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.179621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:34567 (local address 127.11.60.1:39667)
0811 03:12:06.182070 (+ 2449us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:06.182088 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:06.182269 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:06.182874 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:06.182881 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:06.182921 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:06.183398 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.183411 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.184710 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.184717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.186732 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.186749 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.187882 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.187889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.188068 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.188092 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:12:06.188201 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 03:12:06.188344 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2336,"mutex_wait_us":20,"thread_start_us":2177,"threads_started":1}
I20260811 03:12:06.188999 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.182519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:57789 (local address 127.11.60.3:37411)
0811 03:12:06.182708 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.182714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.183352 (+ 638us) server_negotiation.cc:407] Connection header received
0811 03:12:06.183618 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.183622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.183679 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.183777 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:06.184275 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.185325 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.187195 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.187992 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.188134 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.188288 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:06.188608 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 03:12:06.188740 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 03:12:06.189322 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.182404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:37411 (local address 127.11.60.1:57789)
0811 03:12:06.183241 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:06.183260 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:06.183450 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:06.183788 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:06.183792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:06.183814 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:06.184127 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.184135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.185464 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.185471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.187048 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.187060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.188246 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.188252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.188442 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.188466 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:12:06.188569 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 03:12:06.188699 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":457,"threads_started":1}
I20260811 03:12:06.190003 20220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd7d55f3811466787d2b194314c3052" candidate_uuid: "28207ee872d94820bd3be90d5b2cee7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" is_pre_election: true
I20260811 03:12:06.190377 20381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd7d55f3811466787d2b194314c3052" candidate_uuid: "28207ee872d94820bd3be90d5b2cee7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5bc18bf1ac45118599e8390a11229d" is_pre_election: true
I20260811 03:12:06.190677 20220 raft_consensus.cc:2454] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28207ee872d94820bd3be90d5b2cee7b in term 0.
I20260811 03:12:06.190927 20381 raft_consensus.cc:2454] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28207ee872d94820bd3be90d5b2cee7b in term 0.
I20260811 03:12:06.191550 19997 leader_election.cc:302] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28207ee872d94820bd3be90d5b2cee7b, f4fba21ffb2f4dc68f88dbfe7d5d76cc; no voters:
I20260811 03:12:06.192034 20463 raft_consensus.cc:2781] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:12:06.192211 20463 raft_consensus.cc:482] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:12:06.192337 20463 raft_consensus.cc:3037] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:06.194839 20463 raft_consensus.cc:504] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.195468 20463 leader_election.cc:288] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [CANDIDATE]: Term 1 election: Requested vote from peers f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567), 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411)
I20260811 03:12:06.196029 20381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd7d55f3811466787d2b194314c3052" candidate_uuid: "28207ee872d94820bd3be90d5b2cee7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5bc18bf1ac45118599e8390a11229d"
I20260811 03:12:06.196259 20381 raft_consensus.cc:3037] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:06.196269 20220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd7d55f3811466787d2b194314c3052" candidate_uuid: "28207ee872d94820bd3be90d5b2cee7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc"
I20260811 03:12:06.196480 20220 raft_consensus.cc:3037] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:06.198756 20381 raft_consensus.cc:2454] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28207ee872d94820bd3be90d5b2cee7b in term 1.
I20260811 03:12:06.199182 19997 leader_election.cc:302] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28207ee872d94820bd3be90d5b2cee7b, 4e5bc18bf1ac45118599e8390a11229d; no voters:
I20260811 03:12:06.199483 20463 raft_consensus.cc:2781] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:06.200013 20220 raft_consensus.cc:2454] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28207ee872d94820bd3be90d5b2cee7b in term 1.
I20260811 03:12:06.200038 20463 raft_consensus.cc:686] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 1 LEADER]: Becoming Leader. State: Replica: 28207ee872d94820bd3be90d5b2cee7b, State: Running, Role: LEADER
I20260811 03:12:06.200634 20463 consensus_queue.cc:237] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.205449 19904 catalog_manager.cc:5495] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1). New cstate: current_term: 1 leader_uuid: "28207ee872d94820bd3be90d5b2cee7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } health_report { overall_health: UNKNOWN } } }
I20260811 03:12:06.230543 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.225577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.11.60.1:35415)
0811 03:12:06.225775 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.225780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.225798 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:12:06.225888 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.225891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.225985 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.226115 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:12:06.226505 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.227212 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.228512 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.228906 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.228996 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.229380 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:12:06.229976 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:12:06.230085 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:06.230165 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 03:12:06.230233 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 03:12:06.238767 20381 raft_consensus.cc:1261] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Refusing update from remote peer 28207ee872d94820bd3be90d5b2cee7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:06.238768 20220 raft_consensus.cc:1261] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Refusing update from remote peer 28207ee872d94820bd3be90d5b2cee7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:06.239835 20469 consensus_queue.cc:1035] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:06.240235 20463 consensus_queue.cc:1035] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:06.250655 20472 mvcc.cc:205] Tried to move back new op lower bound from 7317167825863712768 to 7317167825722011648. Current Snapshot: MvccSnapshot[applied={T|T < 7317167825863712768}]
W20260811 03:12:06.251482 20433 tablet.cc:2369] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:12:06.253036 20473 mvcc.cc:205] Tried to move back new op lower bound from 7317167825863712768 to 7317167825722011648. Current Snapshot: MvccSnapshot[applied={T|T < 7317167825863712768}]
I20260811 03:12:06.254586 20474 mvcc.cc:205] Tried to move back new op lower bound from 7317167825863712768 to 7317167825722011648. Current Snapshot: MvccSnapshot[applied={T|T < 7317167825863712768}]
I20260811 03:12:06.265560 19904 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:06.265841 19904 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:06.269398 19904 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed: Sending DeleteTablet for 3 replicas of tablet cbd7d55f3811466787d2b194314c3052
I20260811 03:12:06.270577 20051 tablet_service.cc:1514] Processing DeleteTablet for tablet cbd7d55f3811466787d2b194314c3052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:06 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:38780
I20260811 03:12:06.270711 20200 tablet_service.cc:1514] Processing DeleteTablet for tablet cbd7d55f3811466787d2b194314c3052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:06 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:44766
I20260811 03:12:06.270711 20351 tablet_service.cc:1514] Processing DeleteTablet for tablet cbd7d55f3811466787d2b194314c3052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:06 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:55470
I20260811 03:12:06.273684 19904 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:12:06.275141 19904 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:12:06.286701 20481 tablet_replica.cc:330] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: stopping tablet replica
I20260811 03:12:06.286850 20198 tablet_service.cc:1467] Processing CreateTablet for tablet 8896fef4006e483aacfc010f4a31494b (DEFAULT_TABLE table=test-table [id=db6a055e11d74b5a989c698a7f78e7c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:06.287578 20198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8896fef4006e483aacfc010f4a31494b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:06.287531 20481 raft_consensus.cc:2229] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:12:06.288375 20481 raft_consensus.cc:2258] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:06.289279 20483 tablet_replica.cc:330] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: stopping tablet replica
I20260811 03:12:06.290105 20484 tablet_replica.cc:330] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: stopping tablet replica
I20260811 03:12:06.291504 20051 tablet_service.cc:1467] Processing CreateTablet for tablet 8896fef4006e483aacfc010f4a31494b (DEFAULT_TABLE table=test-table [id=db6a055e11d74b5a989c698a7f78e7c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:06.291955 20051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8896fef4006e483aacfc010f4a31494b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:06.293311 20481 ts_tablet_manager.cc:1905] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:06.290289 20350 tablet_service.cc:1467] Processing CreateTablet for tablet 8896fef4006e483aacfc010f4a31494b (DEFAULT_TABLE table=test-table [id=db6a055e11d74b5a989c698a7f78e7c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:06.295197 20458 tablet_bootstrap.cc:492] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Bootstrap starting.
I20260811 03:12:06.295977 20350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8896fef4006e483aacfc010f4a31494b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:06.296525 20484 raft_consensus.cc:2229] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:06.297078 20484 raft_consensus.cc:2258] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:06.297324 20458 tablet_bootstrap.cc:654] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:06.297930 20483 raft_consensus.cc:2229] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:06.298451 20483 raft_consensus.cc:2258] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:06.301718 20484 ts_tablet_manager.cc:1905] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:06.303371 20459 tablet_bootstrap.cc:492] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d: Bootstrap starting.
I20260811 03:12:06.305114 20459 tablet_bootstrap.cc:654] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:06.306936 20457 tablet_bootstrap.cc:492] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: Bootstrap starting.
I20260811 03:12:06.307730 20481 ts_tablet_manager.cc:1918] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:06.307942 20481 log.cc:1196] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/cbd7d55f3811466787d2b194314c3052
I20260811 03:12:06.308732 20481 ts_tablet_manager.cc:1939] T cbd7d55f3811466787d2b194314c3052 P 28207ee872d94820bd3be90d5b2cee7b: Deleting consensus metadata
I20260811 03:12:06.308740 20457 tablet_bootstrap.cc:654] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:06.312163 20483 ts_tablet_manager.cc:1905] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:06.312471 19891 catalog_manager.cc:4841] TS 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415): tablet cbd7d55f3811466787d2b194314c3052 (table test-table [id=106c642f39e24b3e9c99677ed36af1f3]) successfully deleted
I20260811 03:12:06.322013 20483 ts_tablet_manager.cc:1918] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:06.322290 20483 log.cc:1196] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/cbd7d55f3811466787d2b194314c3052
I20260811 03:12:06.323047 20483 ts_tablet_manager.cc:1939] T cbd7d55f3811466787d2b194314c3052 P 4e5bc18bf1ac45118599e8390a11229d: Deleting consensus metadata
I20260811 03:12:06.323396 20484 ts_tablet_manager.cc:1918] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:06.323680 20484 log.cc:1196] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/cbd7d55f3811466787d2b194314c3052
I20260811 03:12:06.324380 20484 ts_tablet_manager.cc:1939] T cbd7d55f3811466787d2b194314c3052 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting consensus metadata
I20260811 03:12:06.324549 20459 tablet_bootstrap.cc:492] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d: No bootstrap required, opened a new log
I20260811 03:12:06.324717 20458 tablet_bootstrap.cc:492] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: No bootstrap required, opened a new log
I20260811 03:12:06.324723 20459 ts_tablet_manager.cc:1397] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.012s
I20260811 03:12:06.324882 20458 ts_tablet_manager.cc:1397] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.012s
I20260811 03:12:06.325302 19890 catalog_manager.cc:4841] TS 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411): tablet cbd7d55f3811466787d2b194314c3052 (table test-table [id=106c642f39e24b3e9c99677ed36af1f3]) successfully deleted
I20260811 03:12:06.325778 20458 raft_consensus.cc:348] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.326001 20458 raft_consensus.cc:374] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:06.326105 20458 raft_consensus.cc:729] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fba21ffb2f4dc68f88dbfe7d5d76cc, State: Initialized, Role: FOLLOWER
I20260811 03:12:06.326529 20457 tablet_bootstrap.cc:492] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: No bootstrap required, opened a new log
I20260811 03:12:06.326712 20457 ts_tablet_manager.cc:1397] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.004s
I20260811 03:12:06.326733 20458 consensus_queue.cc:260] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.327308 20458 ts_tablet_manager.cc:1428] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 03:12:06.327569 20457 raft_consensus.cc:348] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.327801 20457 raft_consensus.cc:374] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:06.327922 20457 raft_consensus.cc:729] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28207ee872d94820bd3be90d5b2cee7b, State: Initialized, Role: FOLLOWER
I20260811 03:12:06.328203 20457 consensus_queue.cc:260] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.328473 19890 catalog_manager.cc:4841] TS f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567): tablet cbd7d55f3811466787d2b194314c3052 (table test-table [id=106c642f39e24b3e9c99677ed36af1f3]) successfully deleted
I20260811 03:12:06.328966 20457 ts_tablet_manager.cc:1428] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 03:12:06.325615 20459 raft_consensus.cc:348] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.329312 20459 raft_consensus.cc:374] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:06.329442 20459 raft_consensus.cc:729] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5bc18bf1ac45118599e8390a11229d, State: Initialized, Role: FOLLOWER
I20260811 03:12:06.329730 20459 consensus_queue.cc:260] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.330487 20459 ts_tablet_manager.cc:1428] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
W20260811 03:12:06.376150 20126 tablet.cc:2369] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:12:06.422041 20276 tablet.cc:2369] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:12:06.450639 20466 raft_consensus.cc:482] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:12:06.450924 20466 raft_consensus.cc:504] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.452981 20466 leader_election.cc:288] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415), 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411)
I20260811 03:12:06.461450 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.453519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:33017 (local address 127.11.60.3:37411)
0811 03:12:06.453711 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.453719 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.454681 (+ 962us) server_negotiation.cc:407] Connection header received
0811 03:12:06.454989 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.454999 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.455089 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.455186 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:06.455842 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.456740 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.459094 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.460371 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.460498 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.460603 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:06.461006 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 03:12:06.461150 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 03:12:06.461500 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.453326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:37411 (local address 127.11.60.2:33017)
0811 03:12:06.454565 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:06.454586 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:06.454813 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:06.455199 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:06.455205 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:06.455287 (+ 82us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:06.455624 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.455637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.456879 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.456886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.458931 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.458946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.460613 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.460619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.460795 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.460843 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 03:12:06.460950 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 03:12:06.461087 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"thread_start_us":879,"threads_started":1}
I20260811 03:12:06.462177 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.453944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:53175 (local address 127.11.60.1:35415)
0811 03:12:06.454129 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.454142 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.455075 (+ 933us) server_negotiation.cc:407] Connection header received
0811 03:12:06.455279 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.455285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.455365 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.455428 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:06.456218 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.457458 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.459784 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.460869 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.460987 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.461085 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:06.461711 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 03:12:06.461853 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 03:12:06.462584 20380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8896fef4006e483aacfc010f4a31494b" candidate_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5bc18bf1ac45118599e8390a11229d" is_pre_election: true
I20260811 03:12:06.462852 20490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.453410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:35415 (local address 127.11.60.2:53175)
0811 03:12:06.454962 (+ 1552us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:06.454980 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:06.455109 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:06.455584 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:06.455589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:06.455611 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:06.456052 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.456064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.457619 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.457625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.459580 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.459592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.461301 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.461307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.461507 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.461548 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 03:12:06.462396 (+ 848us) client_negotiation.cc:240] Negotiation successful
0811 03:12:06.462526 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1394,"thread_start_us":1196,"threads_started":1}
I20260811 03:12:06.462927 20380 raft_consensus.cc:2454] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fba21ffb2f4dc68f88dbfe7d5d76cc in term 0.
I20260811 03:12:06.463483 20148 leader_election.cc:302] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5bc18bf1ac45118599e8390a11229d, f4fba21ffb2f4dc68f88dbfe7d5d76cc; no voters:
I20260811 03:12:06.463936 20466 raft_consensus.cc:2781] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:12:06.463764 20073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8896fef4006e483aacfc010f4a31494b" candidate_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28207ee872d94820bd3be90d5b2cee7b" is_pre_election: true
I20260811 03:12:06.464067 20466 raft_consensus.cc:482] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:12:06.464166 20466 raft_consensus.cc:3037] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:06.464252 20073 raft_consensus.cc:2454] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fba21ffb2f4dc68f88dbfe7d5d76cc in term 0.
I20260811 03:12:06.467532 20466 raft_consensus.cc:504] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.468343 20466 leader_election.cc:288] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [CANDIDATE]: Term 1 election: Requested vote from peers 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415), 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411)
I20260811 03:12:06.468853 20073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8896fef4006e483aacfc010f4a31494b" candidate_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28207ee872d94820bd3be90d5b2cee7b"
I20260811 03:12:06.468871 20380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8896fef4006e483aacfc010f4a31494b" candidate_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5bc18bf1ac45118599e8390a11229d"
I20260811 03:12:06.469105 20073 raft_consensus.cc:3037] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:06.469117 20380 raft_consensus.cc:3037] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:06.471251 20380 raft_consensus.cc:2454] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fba21ffb2f4dc68f88dbfe7d5d76cc in term 1.
I20260811 03:12:06.471683 20148 leader_election.cc:302] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5bc18bf1ac45118599e8390a11229d, f4fba21ffb2f4dc68f88dbfe7d5d76cc; no voters:
I20260811 03:12:06.471977 20466 raft_consensus.cc:2781] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:06.472466 20073 raft_consensus.cc:2454] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fba21ffb2f4dc68f88dbfe7d5d76cc in term 1.
I20260811 03:12:06.472515 20466 raft_consensus.cc:686] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 LEADER]: Becoming Leader. State: Replica: f4fba21ffb2f4dc68f88dbfe7d5d76cc, State: Running, Role: LEADER
I20260811 03:12:06.472963 20466 consensus_queue.cc:237] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.476866 19903 catalog_manager.cc:5495] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc reported cstate change: term changed from 0 to 1, leader changed from <none> to f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2). New cstate: current_term: 1 leader_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } health_report { overall_health: UNKNOWN } } }
Aug 11 03:12:06 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417924, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.62@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:12:06.569303 20498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.553326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40553 (local address 127.0.0.1:39804)
0811 03:12:06.554048 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:06.554066 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:06.554253 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:06.554689 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:06.554693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:06.555473 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:06.555774 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.555784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.556821 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.556829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.557840 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:06.557854 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.558662 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.558668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:06.558833 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.559646 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:06.559676 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:06.562447 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:06.566171 (+ 3724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:06.566188 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:06.566196 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:06.566875 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:06.567280 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:06.567287 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:06.567290 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:06.567453 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:06.568032 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:06.568041 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:06.568251 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 03:12:06.568421 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 03:12:06.568767 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":428,"threads_started":1}
I20260811 03:12:06.569360 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:06.553193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39804 (local address 127.11.60.62:40553)
0811 03:12:06.553493 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:06.553497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:06.554166 (+ 669us) server_negotiation.cc:407] Connection header received
0811 03:12:06.554447 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:06.554452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:06.554567 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:06.554661 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:06.555924 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.556681 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.557987 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:06.558474 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:06.558621 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:06.562696 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:06.562701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:06.562705 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:06.562736 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:06.565973 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:06.567003 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:06.567011 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:06.567015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:06.567163 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:06.567575 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:06.567586 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:06.567589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:06.567895 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:06.568053 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:06.568514 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 03:12:06.568934 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":180,"server-negotiator.queue_time_us":117}
I20260811 03:12:06.576269 19903 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:39804:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 03:12:06.577805 19903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:12:06.586122 20198 tablet_service.cc:1467] Processing CreateTablet for tablet 739ea0d4214d42c891f9f44259d6e078 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=93f61ad6ae9040e889ba9b56f6fd76cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 03:12:06.586184 20051 tablet_service.cc:1467] Processing CreateTablet for tablet 739ea0d4214d42c891f9f44259d6e078 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=93f61ad6ae9040e889ba9b56f6fd76cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 03:12:06.586798 20198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739ea0d4214d42c891f9f44259d6e078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:06.586849 20051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739ea0d4214d42c891f9f44259d6e078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:06.590246 20350 tablet_service.cc:1467] Processing CreateTablet for tablet 739ea0d4214d42c891f9f44259d6e078 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=93f61ad6ae9040e889ba9b56f6fd76cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 03:12:06.590874 20350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739ea0d4214d42c891f9f44259d6e078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:06.594969 20457 tablet_bootstrap.cc:492] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: Bootstrap starting.
I20260811 03:12:06.595767 20458 tablet_bootstrap.cc:492] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Bootstrap starting.
I20260811 03:12:06.597110 20457 tablet_bootstrap.cc:654] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:06.597474 20458 tablet_bootstrap.cc:654] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:06.598717 20459 tablet_bootstrap.cc:492] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: Bootstrap starting.
I20260811 03:12:06.601002 20459 tablet_bootstrap.cc:654] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:06.609728 20457 tablet_bootstrap.cc:492] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: No bootstrap required, opened a new log
I20260811 03:12:06.609968 20457 ts_tablet_manager.cc:1397] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.007s
I20260811 03:12:06.610500 20459 tablet_bootstrap.cc:492] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: No bootstrap required, opened a new log
I20260811 03:12:06.610687 20459 ts_tablet_manager.cc:1397] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.009s
I20260811 03:12:06.610946 20457 raft_consensus.cc:348] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.611229 20457 raft_consensus.cc:374] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:06.611333 20457 raft_consensus.cc:729] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28207ee872d94820bd3be90d5b2cee7b, State: Initialized, Role: FOLLOWER
I20260811 03:12:06.611678 20457 consensus_queue.cc:260] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.611675 20459 raft_consensus.cc:348] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.611999 20459 raft_consensus.cc:374] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:06.612098 20459 raft_consensus.cc:729] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5bc18bf1ac45118599e8390a11229d, State: Initialized, Role: FOLLOWER
I20260811 03:12:06.612185 20463 tablet_replica.cc:439] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } } }
I20260811 03:12:06.612488 20463 tablet_replica.cc:442] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:12:06.612388 20459 consensus_queue.cc:260] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.613083 20459 ts_tablet_manager.cc:1428] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 03:12:06.613461 20464 tablet_replica.cc:439] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } } }
I20260811 03:12:06.613840 20464 tablet_replica.cc:442] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:12:06.616287 20457 ts_tablet_manager.cc:1428] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 03:12:06.617923 20458 tablet_bootstrap.cc:492] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: No bootstrap required, opened a new log
I20260811 03:12:06.618250 20458 ts_tablet_manager.cc:1397] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.010s
I20260811 03:12:06.619143 20458 raft_consensus.cc:348] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.619419 20458 raft_consensus.cc:374] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:06.619541 20458 raft_consensus.cc:729] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fba21ffb2f4dc68f88dbfe7d5d76cc, State: Initialized, Role: FOLLOWER
I20260811 03:12:06.619866 20458 consensus_queue.cc:260] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.620335 20466 tablet_replica.cc:439] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } } }
I20260811 03:12:06.620533 20458 ts_tablet_manager.cc:1428] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 03:12:06.620680 20466 tablet_replica.cc:442] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:12:06.917781 20466 consensus_queue.cc:1035] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:06.924234 20466 consensus_queue.cc:1035] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:06.974941 20466 raft_consensus.cc:482] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:12:06.975175 20466 raft_consensus.cc:504] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.975996 20466 leader_election.cc:288] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415), 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411)
I20260811 03:12:06.976568 20073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739ea0d4214d42c891f9f44259d6e078" candidate_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28207ee872d94820bd3be90d5b2cee7b" is_pre_election: true
I20260811 03:12:06.976629 20380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739ea0d4214d42c891f9f44259d6e078" candidate_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5bc18bf1ac45118599e8390a11229d" is_pre_election: true
I20260811 03:12:06.976874 20073 raft_consensus.cc:2454] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fba21ffb2f4dc68f88dbfe7d5d76cc in term 0.
I20260811 03:12:06.976908 20380 raft_consensus.cc:2454] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fba21ffb2f4dc68f88dbfe7d5d76cc in term 0.
I20260811 03:12:06.977356 20149 leader_election.cc:302] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28207ee872d94820bd3be90d5b2cee7b, f4fba21ffb2f4dc68f88dbfe7d5d76cc; no voters:
I20260811 03:12:06.977664 20466 raft_consensus.cc:2781] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:12:06.977746 20466 raft_consensus.cc:482] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:12:06.977871 20466 raft_consensus.cc:3037] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:06.979960 20466 raft_consensus.cc:504] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.980631 20466 leader_election.cc:288] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [CANDIDATE]: Term 1 election: Requested vote from peers 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415), 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411)
I20260811 03:12:06.981024 20073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739ea0d4214d42c891f9f44259d6e078" candidate_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28207ee872d94820bd3be90d5b2cee7b"
I20260811 03:12:06.981112 20380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739ea0d4214d42c891f9f44259d6e078" candidate_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5bc18bf1ac45118599e8390a11229d"
I20260811 03:12:06.981242 20073 raft_consensus.cc:3037] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:06.981379 20380 raft_consensus.cc:3037] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:06.983405 20380 raft_consensus.cc:2454] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fba21ffb2f4dc68f88dbfe7d5d76cc in term 1.
I20260811 03:12:06.983467 20073 raft_consensus.cc:2454] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fba21ffb2f4dc68f88dbfe7d5d76cc in term 1.
I20260811 03:12:06.983832 20148 leader_election.cc:302] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5bc18bf1ac45118599e8390a11229d, f4fba21ffb2f4dc68f88dbfe7d5d76cc; no voters:
I20260811 03:12:06.984136 20466 raft_consensus.cc:2781] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:06.984310 20466 raft_consensus.cc:686] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 LEADER]: Becoming Leader. State: Replica: f4fba21ffb2f4dc68f88dbfe7d5d76cc, State: Running, Role: LEADER
I20260811 03:12:06.984616 20466 consensus_queue.cc:237] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:06.986052 20491 tablet_replica.cc:439] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: TxnStatusTablet state changed. Reason: New leader f4fba21ffb2f4dc68f88dbfe7d5d76cc. Latest consensus state: current_term: 1 leader_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } } }
I20260811 03:12:06.986331 20491 tablet_replica.cc:442] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: This TxnStatusTablet replica's current role is: LEADER
I20260811 03:12:06.986948 20511 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 03:12:06.987186 20511 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 03:12:06.987774 19904 catalog_manager.cc:5495] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc reported cstate change: term changed from 0 to 1, leader changed from <none> to f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2). New cstate: current_term: 1 leader_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } health_report { overall_health: UNKNOWN } } }
I20260811 03:12:07.076541 20498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.065803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:34567 (local address 127.0.0.1:44774)
0811 03:12:07.066091 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:07.066115 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:07.066325 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:07.070607 (+ 4282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:07.070611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:07.070623 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:12:07.070795 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.070802 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.072368 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.072377 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.073417 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.073430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.074513 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.074517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.074700 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.074774 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.075910 (+ 1136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.075936 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 03:12:07.076032 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 03:12:07.076189 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 03:12:07.076581 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.065962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44774 (local address 127.11.60.2:34567)
0811 03:12:07.070060 (+ 4098us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.070067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.070090 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:12:07.070203 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.070210 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.070363 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.070581 (+ 218us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:12:07.070956 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.072202 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.073596 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.074289 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.074475 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.075018 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.075802 (+ 784us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.075964 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.076060 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.076146 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3939,"thread_start_us":3779,"threads_started":1}
I20260811 03:12:07.080715 20073 raft_consensus.cc:1261] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Refusing update from remote peer f4fba21ffb2f4dc68f88dbfe7d5d76cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:07.080842 20380 raft_consensus.cc:1261] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Refusing update from remote peer f4fba21ffb2f4dc68f88dbfe7d5d76cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:07.081462 20466 consensus_queue.cc:1035] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:07.081797 20491 consensus_queue.cc:1035] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:07.085492 20463 tablet_replica.cc:439] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: TxnStatusTablet state changed. Reason: New leader f4fba21ffb2f4dc68f88dbfe7d5d76cc. Latest consensus state: current_term: 1 leader_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } } }
I20260811 03:12:07.085770 20463 tablet_replica.cc:442] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:12:07.088344 20463 tablet_replica.cc:439] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } } }
I20260811 03:12:07.088626 20463 tablet_replica.cc:442] T 739ea0d4214d42c891f9f44259d6e078 P 28207ee872d94820bd3be90d5b2cee7b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:12:07.095229 20464 tablet_replica.cc:439] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: TxnStatusTablet state changed. Reason: New leader f4fba21ffb2f4dc68f88dbfe7d5d76cc. Latest consensus state: current_term: 1 leader_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } } }
I20260811 03:12:07.095566 20464 tablet_replica.cc:442] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:12:07.096603 20466 tablet_replica.cc:439] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } } }
I20260811 03:12:07.097292 20466 tablet_replica.cc:442] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: This TxnStatusTablet replica's current role is: LEADER
I20260811 03:12:07.099328 20491 tablet_replica.cc:439] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } } }
I20260811 03:12:07.100256 20491 tablet_replica.cc:442] T 739ea0d4214d42c891f9f44259d6e078 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: This TxnStatusTablet replica's current role is: LEADER
I20260811 03:12:07.102288 20464 tablet_replica.cc:439] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } } }
I20260811 03:12:07.102568 20464 tablet_replica.cc:442] T 739ea0d4214d42c891f9f44259d6e078 P 4e5bc18bf1ac45118599e8390a11229d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 03:12:07.112015 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.105776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44786 (local address 127.11.60.2:34567)
0811 03:12:07.105988 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.105995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.106363 (+ 368us) server_negotiation.cc:407] Connection header received
0811 03:12:07.106559 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.106563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.106627 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.106703 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:12:07.107227 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.108030 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.110054 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.110652 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.110794 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.111222 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.111324 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.111392 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.111636 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.111708 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 03:12:07.133204 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.123873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:34567 (local address 127.11.60.2:40591)
0811 03:12:07.126829 (+ 2956us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:07.126851 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:07.127034 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:07.127379 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:07.127395 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:07.127409 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:12:07.127702 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.127714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.128977 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.128983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.130871 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.130887 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.131858 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.131863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.132079 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.132144 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.132623 (+ 479us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.132634 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 03:12:07.132711 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 03:12:07.132827 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2774,"mutex_wait_us":20,"thread_start_us":2641,"threads_started":1}
I20260811 03:12:07.133213 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.123901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:40591 (local address 127.11.60.2:34567)
0811 03:12:07.124094 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.124100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.126990 (+ 2890us) server_negotiation.cc:407] Connection header received
0811 03:12:07.127165 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.127169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.127260 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.127339 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:12:07.127881 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.128821 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.131030 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.131698 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.131824 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.132339 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.132499 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.132595 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.132756 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.132843 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":51}
I20260811 03:12:07.211414 19904 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:07.211679 19904 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:07.215166 19904 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed: Sending DeleteTablet for 3 replicas of tablet 8896fef4006e483aacfc010f4a31494b
I20260811 03:12:07.216346 20200 tablet_service.cc:1514] Processing DeleteTablet for tablet 8896fef4006e483aacfc010f4a31494b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:07 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:44766
I20260811 03:12:07.216795 20350 tablet_service.cc:1514] Processing DeleteTablet for tablet 8896fef4006e483aacfc010f4a31494b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:07 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:55470
I20260811 03:12:07.216809 20051 tablet_service.cc:1514] Processing DeleteTablet for tablet 8896fef4006e483aacfc010f4a31494b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:07 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:38780
I20260811 03:12:07.217934 20531 tablet_replica.cc:330] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: stopping tablet replica
I20260811 03:12:07.218168 20530 tablet_replica.cc:330] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: stopping tablet replica
I20260811 03:12:07.218401 20530 raft_consensus.cc:2229] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:12:07.218353 20531 raft_consensus.cc:2229] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:07.219003 20531 raft_consensus.cc:2258] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:07.219079 20530 raft_consensus.cc:2258] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:07.220422 20531 ts_tablet_manager.cc:1905] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:07.227807 20531 ts_tablet_manager.cc:1918] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:12:07.227973 20531 log.cc:1196] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/8896fef4006e483aacfc010f4a31494b
I20260811 03:12:07.228492 20531 ts_tablet_manager.cc:1939] T 8896fef4006e483aacfc010f4a31494b P 28207ee872d94820bd3be90d5b2cee7b: Deleting consensus metadata
I20260811 03:12:07.229633 19891 catalog_manager.cc:4841] TS 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415): tablet 8896fef4006e483aacfc010f4a31494b (table test-table [id=db6a055e11d74b5a989c698a7f78e7c2]) successfully deleted
I20260811 03:12:07.230455 20530 ts_tablet_manager.cc:1905] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:07.233611 20532 tablet_replica.cc:330] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d: stopping tablet replica
I20260811 03:12:07.233860 20532 raft_consensus.cc:2229] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:07.234117 20532 raft_consensus.cc:2258] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:07.235190 20532 ts_tablet_manager.cc:1905] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:07.237934 20530 ts_tablet_manager.cc:1918] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:12:07.238135 20530 log.cc:1196] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/8896fef4006e483aacfc010f4a31494b
I20260811 03:12:07.238768 20530 ts_tablet_manager.cc:1939] T 8896fef4006e483aacfc010f4a31494b P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting consensus metadata
I20260811 03:12:07.240106 19890 catalog_manager.cc:4841] TS f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567): tablet 8896fef4006e483aacfc010f4a31494b (table test-table [id=db6a055e11d74b5a989c698a7f78e7c2]) successfully deleted
I20260811 03:12:07.243214 20532 ts_tablet_manager.cc:1918] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:12:07.243379 20532 log.cc:1196] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/8896fef4006e483aacfc010f4a31494b
I20260811 03:12:07.243836 20532 ts_tablet_manager.cc:1939] T 8896fef4006e483aacfc010f4a31494b P 4e5bc18bf1ac45118599e8390a11229d: Deleting consensus metadata
I20260811 03:12:07.245121 19890 catalog_manager.cc:4841] TS 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411): tablet 8896fef4006e483aacfc010f4a31494b (table test-table [id=db6a055e11d74b5a989c698a7f78e7c2]) successfully deleted
Aug 11 03:12:07 dist-test-slave-tm1g krb5kdc[19855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 03:12:07 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:07 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 03:12:07.292320 20539 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:07.280254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.11.60.62:40553)
0811 03:12:07.280923 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.280930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.281183 (+ 253us) server_negotiation.cc:407] Connection header received
0811 03:12:07.281410 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.281414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.281483 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.281609 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:07.283141 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.284039 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.285407 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.285841 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.285980 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.291460 (+ 5480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39808: BlockingRecv error: recv got EOF from 127.0.0.1:39808 (error 108)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":322,"threads_started":1}
Aug 11 03:12:07 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM
I20260811 03:12:07.311488 20539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.295110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.11.60.62:40553)
0811 03:12:07.295372 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.295381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.295865 (+ 484us) server_negotiation.cc:407] Connection header received
0811 03:12:07.296072 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.296077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.296169 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.296275 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:07.297489 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.298227 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.299501 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.299917 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.300052 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.304392 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:07.304399 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:07.304405 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:07.304460 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:07.308216 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:07.308944 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:07.308959 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:07.308965 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:07.309140 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:07.309666 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:07.309672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:07.309677 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:07.310090 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:07.310418 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.310837 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.311117 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 03:12:07.315369 19904 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:12:07.316076 19904 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:12:07.323328 20051 tablet_service.cc:1467] Processing CreateTablet for tablet c6941100ab60496690949b4ef13b1407 (DEFAULT_TABLE table=test-table [id=57a799a761114cf29c83fdea4aa83d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:07.323429 20200 tablet_service.cc:1467] Processing CreateTablet for tablet c6941100ab60496690949b4ef13b1407 (DEFAULT_TABLE table=test-table [id=57a799a761114cf29c83fdea4aa83d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:07.323848 20051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6941100ab60496690949b4ef13b1407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:07.323911 20200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6941100ab60496690949b4ef13b1407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:07.324083 20350 tablet_service.cc:1467] Processing CreateTablet for tablet c6941100ab60496690949b4ef13b1407 (DEFAULT_TABLE table=test-table [id=57a799a761114cf29c83fdea4aa83d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:07.324541 20350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6941100ab60496690949b4ef13b1407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:07.336630 20546 tablet_bootstrap.cc:492] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b: Bootstrap starting.
I20260811 03:12:07.337271 20548 tablet_bootstrap.cc:492] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d: Bootstrap starting.
I20260811 03:12:07.339340 20547 tablet_bootstrap.cc:492] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Bootstrap starting.
I20260811 03:12:07.341090 20546 tablet_bootstrap.cc:654] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:07.341539 20547 tablet_bootstrap.cc:654] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:07.342252 20548 tablet_bootstrap.cc:654] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:07.344547 20547 tablet_bootstrap.cc:492] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: No bootstrap required, opened a new log
I20260811 03:12:07.344730 20547 ts_tablet_manager.cc:1397] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 03:12:07.345729 20546 tablet_bootstrap.cc:492] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b: No bootstrap required, opened a new log
I20260811 03:12:07.345711 20547 raft_consensus.cc:348] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.345903 20546 ts_tablet_manager.cc:1397] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260811 03:12:07.346012 20547 raft_consensus.cc:374] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:07.346124 20547 raft_consensus.cc:729] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fba21ffb2f4dc68f88dbfe7d5d76cc, State: Initialized, Role: FOLLOWER
I20260811 03:12:07.346442 20547 consensus_queue.cc:260] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.347103 20547 ts_tablet_manager.cc:1428] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 03:12:07.347079 20546 raft_consensus.cc:348] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.347421 20546 raft_consensus.cc:374] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:07.347534 20546 raft_consensus.cc:729] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28207ee872d94820bd3be90d5b2cee7b, State: Initialized, Role: FOLLOWER
I20260811 03:12:07.347805 20546 consensus_queue.cc:260] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.348815 20548 tablet_bootstrap.cc:492] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d: No bootstrap required, opened a new log
I20260811 03:12:07.349092 20548 ts_tablet_manager.cc:1397] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.002s
I20260811 03:12:07.350340 20548 raft_consensus.cc:348] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.350680 20548 raft_consensus.cc:374] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:07.350836 20548 raft_consensus.cc:729] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5bc18bf1ac45118599e8390a11229d, State: Initialized, Role: FOLLOWER
I20260811 03:12:07.351186 20548 consensus_queue.cc:260] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.351845 20546 ts_tablet_manager.cc:1428] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 03:12:07.354157 20548 ts_tablet_manager.cc:1428] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 03:12:07.520879 20463 raft_consensus.cc:482] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:12:07.521131 20463 raft_consensus.cc:504] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.522024 20463 leader_election.cc:288] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567), 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411)
I20260811 03:12:07.522706 20220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6941100ab60496690949b4ef13b1407" candidate_uuid: "28207ee872d94820bd3be90d5b2cee7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" is_pre_election: true
I20260811 03:12:07.522799 20380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6941100ab60496690949b4ef13b1407" candidate_uuid: "28207ee872d94820bd3be90d5b2cee7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5bc18bf1ac45118599e8390a11229d" is_pre_election: true
I20260811 03:12:07.523048 20220 raft_consensus.cc:2454] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28207ee872d94820bd3be90d5b2cee7b in term 0.
I20260811 03:12:07.523058 20380 raft_consensus.cc:2454] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28207ee872d94820bd3be90d5b2cee7b in term 0.
I20260811 03:12:07.523607 19997 leader_election.cc:302] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28207ee872d94820bd3be90d5b2cee7b, f4fba21ffb2f4dc68f88dbfe7d5d76cc; no voters:
I20260811 03:12:07.523921 20463 raft_consensus.cc:2781] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:12:07.524078 20463 raft_consensus.cc:482] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:12:07.524185 20463 raft_consensus.cc:3037] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:07.527594 20463 raft_consensus.cc:504] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.528389 20463 leader_election.cc:288] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [CANDIDATE]: Term 1 election: Requested vote from peers f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567), 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411)
I20260811 03:12:07.528983 20380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6941100ab60496690949b4ef13b1407" candidate_uuid: "28207ee872d94820bd3be90d5b2cee7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5bc18bf1ac45118599e8390a11229d"
I20260811 03:12:07.529006 20220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6941100ab60496690949b4ef13b1407" candidate_uuid: "28207ee872d94820bd3be90d5b2cee7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc"
I20260811 03:12:07.529248 20380 raft_consensus.cc:3037] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:07.529256 20220 raft_consensus.cc:3037] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:07.531451 20380 raft_consensus.cc:2454] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28207ee872d94820bd3be90d5b2cee7b in term 1.
I20260811 03:12:07.531973 19997 leader_election.cc:302] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28207ee872d94820bd3be90d5b2cee7b, 4e5bc18bf1ac45118599e8390a11229d; no voters:
I20260811 03:12:07.532289 20463 raft_consensus.cc:2781] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:07.532778 20463 raft_consensus.cc:686] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 1 LEADER]: Becoming Leader. State: Replica: 28207ee872d94820bd3be90d5b2cee7b, State: Running, Role: LEADER
I20260811 03:12:07.532783 20220 raft_consensus.cc:2454] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28207ee872d94820bd3be90d5b2cee7b in term 1.
I20260811 03:12:07.533111 20463 consensus_queue.cc:237] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.536098 19904 catalog_manager.cc:5495] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1). New cstate: current_term: 1 leader_uuid: "28207ee872d94820bd3be90d5b2cee7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } health_report { overall_health: UNKNOWN } } }
I20260811 03:12:07.585136 20555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.578390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38812 (local address 127.11.60.1:35415)
0811 03:12:07.579156 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.579163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.579190 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:12:07.579270 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.579276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.579370 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.579492 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:12:07.579858 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.581190 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.582653 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.583448 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.583623 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.584104 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.584255 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.584364 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.584619 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.584711 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":432,"threads_started":1}
I20260811 03:12:07.588248 20220 raft_consensus.cc:1261] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Refusing update from remote peer 28207ee872d94820bd3be90d5b2cee7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:07.588292 20380 raft_consensus.cc:1261] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Refusing update from remote peer 28207ee872d94820bd3be90d5b2cee7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:07.589018 20552 consensus_queue.cc:1035] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:07.589398 20463 consensus_queue.cc:1035] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:07.602037 19903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:07.602388 19903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:07.607038 19903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed: Sending DeleteTablet for 3 replicas of tablet c6941100ab60496690949b4ef13b1407
I20260811 03:12:07.608144 20051 tablet_service.cc:1514] Processing DeleteTablet for tablet c6941100ab60496690949b4ef13b1407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:07 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:38780
I20260811 03:12:07.608474 20531 tablet_replica.cc:330] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b: stopping tablet replica
I20260811 03:12:07.608435 20200 tablet_service.cc:1514] Processing DeleteTablet for tablet c6941100ab60496690949b4ef13b1407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:07 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:44766
I20260811 03:12:07.608541 20050 tablet_service.cc:1514] Processing DeleteTablet for tablet c6941100ab60496690949b4ef13b1407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:07 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:38780
I20260811 03:12:07.608717 20531 raft_consensus.cc:2229] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:12:07.609170 20530 tablet_replica.cc:330] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: stopping tablet replica
I20260811 03:12:07.609391 20530 raft_consensus.cc:2229] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:07.609472 20350 tablet_service.cc:1514] Processing DeleteTablet for tablet c6941100ab60496690949b4ef13b1407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:07 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:55470
I20260811 03:12:07.609627 20530 raft_consensus.cc:2258] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:07.609794 20532 tablet_replica.cc:330] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d: stopping tablet replica
I20260811 03:12:07.610005 20532 raft_consensus.cc:2229] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:07.610289 20532 raft_consensus.cc:2258] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:07.610591 19905 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:12:07.611374 19905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:12:07.611369 20530 ts_tablet_manager.cc:1905] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:07.612593 20532 ts_tablet_manager.cc:1905] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:07.619359 20530 ts_tablet_manager.cc:1918] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:07.619549 20530 log.cc:1196] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/c6941100ab60496690949b4ef13b1407
I20260811 03:12:07.620105 20530 ts_tablet_manager.cc:1939] T c6941100ab60496690949b4ef13b1407 P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting consensus metadata
I20260811 03:12:07.621493 19890 catalog_manager.cc:4841] TS f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567): tablet c6941100ab60496690949b4ef13b1407 (table test-table [id=57a799a761114cf29c83fdea4aa83d31]) successfully deleted
I20260811 03:12:07.622570 20531 raft_consensus.cc:2258] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:07.623610 20531 ts_tablet_manager.cc:1905] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:07.624444 20532 ts_tablet_manager.cc:1918] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:07.624605 20532 log.cc:1196] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/c6941100ab60496690949b4ef13b1407
I20260811 03:12:07.625068 20532 ts_tablet_manager.cc:1939] T c6941100ab60496690949b4ef13b1407 P 4e5bc18bf1ac45118599e8390a11229d: Deleting consensus metadata
I20260811 03:12:07.628526 19890 catalog_manager.cc:4841] TS 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411): tablet c6941100ab60496690949b4ef13b1407 (table test-table [id=57a799a761114cf29c83fdea4aa83d31]) successfully deleted
I20260811 03:12:07.630419 20050 tablet_service.cc:1467] Processing CreateTablet for tablet 97e8245d13ba42d4903369d53ee4721c (DEFAULT_TABLE table=test-table [id=758506de1e6d4a1e8835c62c1368653d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:07.630378 20200 tablet_service.cc:1467] Processing CreateTablet for tablet 97e8245d13ba42d4903369d53ee4721c (DEFAULT_TABLE table=test-table [id=758506de1e6d4a1e8835c62c1368653d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:07.630949 20200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e8245d13ba42d4903369d53ee4721c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:07.631155 20350 tablet_service.cc:1467] Processing CreateTablet for tablet 97e8245d13ba42d4903369d53ee4721c (DEFAULT_TABLE table=test-table [id=758506de1e6d4a1e8835c62c1368653d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:07.631546 20350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e8245d13ba42d4903369d53ee4721c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:07.634534 20531 ts_tablet_manager.cc:1918] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:07.634760 20531 log.cc:1196] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/c6941100ab60496690949b4ef13b1407
I20260811 03:12:07.635308 20531 ts_tablet_manager.cc:1939] T c6941100ab60496690949b4ef13b1407 P 28207ee872d94820bd3be90d5b2cee7b: Deleting consensus metadata
I20260811 03:12:07.635764 20050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e8245d13ba42d4903369d53ee4721c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:07.638424 20547 tablet_bootstrap.cc:492] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Bootstrap starting.
I20260811 03:12:07.639420 20548 tablet_bootstrap.cc:492] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d: Bootstrap starting.
I20260811 03:12:07.640507 20547 tablet_bootstrap.cc:654] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:07.641140 20548 tablet_bootstrap.cc:654] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:07.643451 20546 tablet_bootstrap.cc:492] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b: Bootstrap starting.
I20260811 03:12:07.645237 20546 tablet_bootstrap.cc:654] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:07.649519 20548 tablet_bootstrap.cc:492] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d: No bootstrap required, opened a new log
I20260811 03:12:07.649664 20547 tablet_bootstrap.cc:492] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc: No bootstrap required, opened a new log
I20260811 03:12:07.649700 20548 ts_tablet_manager.cc:1397] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260811 03:12:07.649897 20547 ts_tablet_manager.cc:1397] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260811 03:12:07.650594 20548 raft_consensus.cc:348] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.650870 20548 raft_consensus.cc:374] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:07.650775 20547 raft_consensus.cc:348] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.650952 20548 raft_consensus.cc:729] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5bc18bf1ac45118599e8390a11229d, State: Initialized, Role: FOLLOWER
I20260811 03:12:07.651062 20547 raft_consensus.cc:374] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:07.651162 20547 raft_consensus.cc:729] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fba21ffb2f4dc68f88dbfe7d5d76cc, State: Initialized, Role: FOLLOWER
I20260811 03:12:07.651216 20548 consensus_queue.cc:260] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.651445 20547 consensus_queue.cc:260] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.651752 20548 ts_tablet_manager.cc:1428] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 03:12:07.652201 20547 ts_tablet_manager.cc:1428] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 03:12:07.654937 20546 tablet_bootstrap.cc:492] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b: No bootstrap required, opened a new log
I20260811 03:12:07.655248 20546 ts_tablet_manager.cc:1397] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.001s
I20260811 03:12:07.656113 20546 raft_consensus.cc:348] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.656483 20546 raft_consensus.cc:374] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:07.656584 20546 raft_consensus.cc:729] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28207ee872d94820bd3be90d5b2cee7b, State: Initialized, Role: FOLLOWER
I20260811 03:12:07.656967 20546 consensus_queue.cc:260] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.657984 19891 catalog_manager.cc:4841] TS 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415): tablet c6941100ab60496690949b4ef13b1407 (table test-table [id=57a799a761114cf29c83fdea4aa83d31]) successfully deleted
I20260811 03:12:07.658179 20546 ts_tablet_manager.cc:1428] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
W20260811 03:12:07.658485 19891 catalog_manager.cc:4808] TS 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415): delete failed for tablet c6941100ab60496690949b4ef13b1407 because the tablet was not found. No further retry: Not found: Tablet not found: c6941100ab60496690949b4ef13b1407
I20260811 03:12:07.737742 20464 raft_consensus.cc:482] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:12:07.738025 20464 raft_consensus.cc:504] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.740319 20464 leader_election.cc:288] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415), f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567)
I20260811 03:12:07.758882 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.740919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:34567 (local address 127.11.60.3:48405)
0811 03:12:07.750697 (+ 9778us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:07.750723 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:07.750945 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:07.751617 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:07.751622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:07.751653 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:07.752051 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.752067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.753910 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.753916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.756109 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.756121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.757892 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.757901 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.758101 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.758149 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 03:12:07.758272 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 03:12:07.758464 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9600,"thread_start_us":12860,"threads_started":1}
I20260811 03:12:07.758877 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.746421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:48405 (local address 127.11.60.2:34567)
0811 03:12:07.750675 (+ 4254us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.750682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.751085 (+ 403us) server_negotiation.cc:407] Connection header received
0811 03:12:07.751179 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.751185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.751316 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.751430 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:07.752517 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.753665 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.756281 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.757655 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.757792 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.757959 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.758334 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.758490 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4093,"thread_start_us":3616,"threads_started":1}
I20260811 03:12:07.760102 20220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e8245d13ba42d4903369d53ee4721c" candidate_uuid: "4e5bc18bf1ac45118599e8390a11229d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" is_pre_election: true
I20260811 03:12:07.760406 20220 raft_consensus.cc:2454] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e5bc18bf1ac45118599e8390a11229d in term 0.
I20260811 03:12:07.761044 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.741138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:35415 (local address 127.11.60.3:54553)
0811 03:12:07.754135 (+ 12997us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:07.754159 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:07.754284 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:07.754683 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:07.754689 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:07.754722 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:07.755148 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.755164 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.756628 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.756633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.758631 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.758646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.760235 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.760241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.760444 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.760469 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 03:12:07.760581 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 03:12:07.760710 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12862,"thread_start_us":12284,"threads_started":1}
I20260811 03:12:07.761065 20555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.746912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:54553 (local address 127.11.60.1:35415)
0811 03:12:07.747170 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.747177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.754334 (+ 7157us) server_negotiation.cc:407] Connection header received
0811 03:12:07.754437 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.754443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.754537 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.754647 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:07.755336 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.756478 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.758820 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.760023 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.760168 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.760281 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.760671 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.760806 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 03:12:07.761125 20300 leader_election.cc:302] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5bc18bf1ac45118599e8390a11229d, f4fba21ffb2f4dc68f88dbfe7d5d76cc; no voters:
I20260811 03:12:07.761602 20464 raft_consensus.cc:2781] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:12:07.761730 20464 raft_consensus.cc:482] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:12:07.761828 20464 raft_consensus.cc:3037] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:07.762032 20073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e8245d13ba42d4903369d53ee4721c" candidate_uuid: "4e5bc18bf1ac45118599e8390a11229d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28207ee872d94820bd3be90d5b2cee7b" is_pre_election: true
I20260811 03:12:07.762369 20073 raft_consensus.cc:2454] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e5bc18bf1ac45118599e8390a11229d in term 0.
I20260811 03:12:07.764583 20464 raft_consensus.cc:504] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.765226 20464 leader_election.cc:288] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [CANDIDATE]: Term 1 election: Requested vote from peers 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415), f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567)
I20260811 03:12:07.765798 20073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e8245d13ba42d4903369d53ee4721c" candidate_uuid: "4e5bc18bf1ac45118599e8390a11229d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28207ee872d94820bd3be90d5b2cee7b"
I20260811 03:12:07.765844 20220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e8245d13ba42d4903369d53ee4721c" candidate_uuid: "4e5bc18bf1ac45118599e8390a11229d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc"
I20260811 03:12:07.766074 20073 raft_consensus.cc:3037] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:07.766132 20220 raft_consensus.cc:3037] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:07.768347 20073 raft_consensus.cc:2454] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e5bc18bf1ac45118599e8390a11229d in term 1.
I20260811 03:12:07.769613 20220 raft_consensus.cc:2454] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e5bc18bf1ac45118599e8390a11229d in term 1.
I20260811 03:12:07.770093 20300 leader_election.cc:302] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5bc18bf1ac45118599e8390a11229d, f4fba21ffb2f4dc68f88dbfe7d5d76cc; no voters:
I20260811 03:12:07.770464 20464 raft_consensus.cc:2781] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:07.771034 20464 raft_consensus.cc:686] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 1 LEADER]: Becoming Leader. State: Replica: 4e5bc18bf1ac45118599e8390a11229d, State: Running, Role: LEADER
I20260811 03:12:07.771487 20464 consensus_queue.cc:237] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } }
I20260811 03:12:07.775574 19904 catalog_manager.cc:5495] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3). New cstate: current_term: 1 leader_uuid: "4e5bc18bf1ac45118599e8390a11229d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e5bc18bf1ac45118599e8390a11229d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37411 } health_report { overall_health: HEALTHY } } }
I20260811 03:12:07.829568 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.822851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55476 (local address 127.11.60.3:37411)
0811 03:12:07.823469 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.823473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.823492 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:12:07.823582 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.823585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.823667 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.823858 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:12:07.824332 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.825130 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.826600 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.827043 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.827188 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.827765 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.828548 (+ 783us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.829174 (+ 626us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.829226 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.829293 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":363,"threads_started":1}
I20260811 03:12:07.848380 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.841882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:43885 (local address 127.11.60.2:34567)
0811 03:12:07.842096 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.842102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.842553 (+ 451us) server_negotiation.cc:407] Connection header received
0811 03:12:07.842748 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.842752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.842828 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.842915 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:12:07.843602 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.844371 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.846513 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.847099 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.847217 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.847746 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.847846 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.847915 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.848078 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.848154 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 03:12:07.848605 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.841683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:34567 (local address 127.11.60.3:43885)
0811 03:12:07.842414 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:07.842436 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:07.842611 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:07.842964 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:07.842970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:07.842983 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:12:07.843436 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.843449 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.844528 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.844534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.846348 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.846364 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.847301 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.847308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.847491 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.847572 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.847951 (+ 379us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.847963 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 03:12:07.848058 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 03:12:07.848213 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":294,"threads_started":1}
I20260811 03:12:07.859567 20220 raft_consensus.cc:1261] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Refusing update from remote peer 4e5bc18bf1ac45118599e8390a11229d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:07.859648 20073 raft_consensus.cc:1261] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Refusing update from remote peer 4e5bc18bf1ac45118599e8390a11229d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:07.860466 20566 consensus_queue.cc:1035] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [LEADER]: Connected to new peer: Peer: permanent_uuid: "28207ee872d94820bd3be90d5b2cee7b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 35415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:07.861053 20464 consensus_queue.cc:1035] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4fba21ffb2f4dc68f88dbfe7d5d76cc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:07.913077 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.903844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:35895 (local address 127.11.60.3:37411)
0811 03:12:07.904083 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.904093 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.904844 (+ 751us) server_negotiation.cc:407] Connection header received
0811 03:12:07.905734 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.905742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.905887 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.905962 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:12:07.907358 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.908251 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.909920 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.910641 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.910720 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.912163 (+ 1443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.912288 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.912342 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.912733 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.912824 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 03:12:07.913702 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.903727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:37411 (local address 127.11.60.2:35895)
0811 03:12:07.904723 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:07.904749 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:07.905145 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:07.906558 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:07.906566 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:07.906580 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 03:12:07.907178 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.907193 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.908395 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.908401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.909730 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:07.909744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.911356 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.911363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:07.911921 (+ 558us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.911977 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 03:12:07.912576 (+ 599us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 03:12:07.912589 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 03:12:07.913293 (+ 704us) client_negotiation.cc:240] Negotiation successful
0811 03:12:07.913419 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":651,"threads_started":1}
I20260811 03:12:07.953399 19905 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:07.953701 19905 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:07.957561 19905 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59aa121f37364f5b8905751cfec5e7ed: Sending DeleteTablet for 3 replicas of tablet 97e8245d13ba42d4903369d53ee4721c
I20260811 03:12:07.958734 20050 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e8245d13ba42d4903369d53ee4721c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:07 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:38780
I20260811 03:12:07.959084 20531 tablet_replica.cc:330] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b: stopping tablet replica
I20260811 03:12:07.959071 20350 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e8245d13ba42d4903369d53ee4721c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:07 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:55470
I20260811 03:12:07.959295 20531 raft_consensus.cc:2229] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:07.959359 20532 tablet_replica.cc:330] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d: stopping tablet replica
I20260811 03:12:07.959522 20531 raft_consensus.cc:2258] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:07.959551 20532 raft_consensus.cc:2229] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:12:07.959918 20200 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e8245d13ba42d4903369d53ee4721c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:07 UTC) from {username='oryx', principal='oryx/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:44766
I20260811 03:12:07.960103 20532 raft_consensus.cc:2258] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:07.960438 20530 tablet_replica.cc:330] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc: stopping tablet replica
I20260811 03:12:07.960639 20530 raft_consensus.cc:2229] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:07.960783 20531 ts_tablet_manager.cc:1905] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:07.960862 20530 raft_consensus.cc:2258] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:07.962216 20530 ts_tablet_manager.cc:1905] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:07.962478 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 19979
I20260811 03:12:07.968981 20530 ts_tablet_manager.cc:1918] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:12:07.969147 20530 log.cc:1196] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/97e8245d13ba42d4903369d53ee4721c
I20260811 03:12:07.969661 20530 ts_tablet_manager.cc:1939] T 97e8245d13ba42d4903369d53ee4721c P f4fba21ffb2f4dc68f88dbfe7d5d76cc: Deleting consensus metadata
I20260811 03:12:07.970916 19890 catalog_manager.cc:4841] TS f4fba21ffb2f4dc68f88dbfe7d5d76cc (127.11.60.2:34567): tablet 97e8245d13ba42d4903369d53ee4721c (table test-table [id=758506de1e6d4a1e8835c62c1368653d]) successfully deleted
I20260811 03:12:07.971189 20531 ts_tablet_manager.cc:1918] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:12:07.971397 20531 log.cc:1196] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/97e8245d13ba42d4903369d53ee4721c
I20260811 03:12:07.971964 20531 ts_tablet_manager.cc:1939] T 97e8245d13ba42d4903369d53ee4721c P 28207ee872d94820bd3be90d5b2cee7b: Deleting consensus metadata
I20260811 03:12:07.972154 20532 ts_tablet_manager.cc:1905] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:07.973397 19891 catalog_manager.cc:4841] TS 28207ee872d94820bd3be90d5b2cee7b (127.11.60.1:35415): tablet 97e8245d13ba42d4903369d53ee4721c (table test-table [id=758506de1e6d4a1e8835c62c1368653d]) successfully deleted
I20260811 03:12:07.979115 20532 ts_tablet_manager.cc:1918] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 03:12:07.979245 20532 log.cc:1196] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/97e8245d13ba42d4903369d53ee4721c
I20260811 03:12:07.979671 20532 ts_tablet_manager.cc:1939] T 97e8245d13ba42d4903369d53ee4721c P 4e5bc18bf1ac45118599e8390a11229d: Deleting consensus metadata
I20260811 03:12:07.980984 19890 catalog_manager.cc:4841] TS 4e5bc18bf1ac45118599e8390a11229d (127.11.60.3:37411): tablet 97e8245d13ba42d4903369d53ee4721c (table test-table [id=758506de1e6d4a1e8835c62c1368653d]) successfully deleted
Aug 11 03:12:07 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.1@KRBTEST.COM
I20260811 03:12:07.991482 20555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:07.963691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38828 (local address 127.11.60.1:35415)
0811 03:12:07.963870 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:07.963878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:07.973729 (+ 9851us) server_negotiation.cc:407] Connection header received
0811 03:12:07.973921 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:07.973925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:07.974007 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:07.974120 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:07.975834 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.976963 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.978868 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:07.979545 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:07.979687 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:07.984194 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:07.984216 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:07.984236 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:07.984291 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:07.988331 (+ 4040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:07.989094 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:07.989104 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:07.989108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:07.989282 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:07.989657 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:07.989662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:07.989666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:07.990178 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:07.990530 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:07.990806 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 03:12:07.991089 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 03:12:07.992231 20120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:08.122190 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 19979
I20260811 03:12:08.157966 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 20129
Aug 11 03:12:08 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.2@KRBTEST.COM
I20260811 03:12:08.176296 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:08.158984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.11.60.2:34567)
0811 03:12:08.159172 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:08.159178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:08.159200 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:12:08.159271 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:08.159276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:08.159371 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:08.159488 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:08.160937 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:08.162071 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:08.163697 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:08.164387 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:08.164520 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:08.168821 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:08.168841 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:08.168882 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:08.168940 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:08.173183 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:08.173893 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:08.173903 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:08.173906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:08.174071 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:08.174546 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:08.174551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:08.174555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:08.175058 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:08.175385 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:08.175662 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 03:12:08.175902 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 03:12:08.176966 20270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:08.337186 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 20129
I20260811 03:12:08.369661 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 20279
Aug 11 03:12:08 dist-test-slave-tm1g krb5kdc[19855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.60.3@KRBTEST.COM
I20260811 03:12:08.390990 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:08.370513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.11.60.3:37411)
0811 03:12:08.370740 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:08.370745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:08.370856 (+ 111us) server_negotiation.cc:407] Connection header received
0811 03:12:08.371092 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:08.371099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:08.371212 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:08.371345 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:08.373155 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:08.373945 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:08.375866 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:08.376308 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:08.376403 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:08.383019 (+ 6616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:08.383056 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:08.383102 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:08.383167 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:08.387565 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:08.388466 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:08.388475 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:08.388478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:08.388634 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:08.389101 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:08.389105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:08.389108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:08.389540 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:08.389952 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:08.390342 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 03:12:08.390632 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:12:08.391805 20426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:08.530589 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 20279
I20260811 03:12:08.552901 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 19870
I20260811 03:12:08.554281 19947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:08.675045 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 19870
2026-08-11T03:12:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6434 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:12:08 dist-test-slave-tm1g krb5kdc[20590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:12:08 dist-test-slave-tm1g krb5kdc[20590](info): set up 2 sockets
Aug 11 03:12:08 dist-test-slave-tm1g krb5kdc[20590](info): commencing operation
krb5kdc: starting...
W20260811 03:12:09.873179 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.002s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:12:10 dist-test-slave-tm1g krb5kdc[20590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:12:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:12:10.150646 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40361
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:40361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:10.491890 20604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:10.492208 20604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:10.492304 20604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:10.499800 20604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:12:10.499928 20604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:10.499976 20604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:10.500034 20604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:12:10.500079 20604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:12:10.510278 20604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:40361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:10.511700 20604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:10.513059 20604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:10.525506 20610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:10.525437 20613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:10.525480 20611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:10.526538 20604 server_base.cc:1034] running on GCE node
I20260811 03:12:10.527418 20604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:10.529044 20604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:10.530259 20604 hybrid_clock.cc:648] HybridClock initialized: now 1786417930530221 us; error 32 us; skew 500 ppm
I20260811 03:12:10.530793 20604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T03:12:10Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1924 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:12:10 dist-test-slave-tm1g krb5kdc[20619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:12:10 dist-test-slave-tm1g krb5kdc[20619](info): set up 2 sockets
Aug 11 03:12:10 dist-test-slave-tm1g krb5kdc[20619](info): commencing operation
krb5kdc: starting...
W20260811 03:12:11.788357 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:12:11 dist-test-slave-tm1g krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:12:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:12:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:12:12.077859 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:39337
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:39337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:12.419977 20632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:12.420285 20632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:12.420378 20632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:12.427804 20632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:12:12.427922 20632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:12.427968 20632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:12.428027 20632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:12:12.428072 20632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:12:12.438076 20632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:39337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:39337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:12.439498 20632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:12.440806 20632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:12.453246 20637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:12.453181 20640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:12.453703 20638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:12.454468 20632 server_base.cc:1034] running on GCE node
I20260811 03:12:12.455312 20632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:12.456763 20632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:12.458004 20632 hybrid_clock.cc:648] HybridClock initialized: now 1786417932457933 us; error 42 us; skew 500 ppm
I20260811 03:12:12.458437 20632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:12 dist-test-slave-tm1g krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417932, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:12.461642 20632 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:12:12.463335 20632 webserver.cc:460] Webserver started at http://127.11.60.62:34487/ using document root <none> and password file <none>
I20260811 03:12:12.464087 20632 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:12.464221 20632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:12.464557 20632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:12.467712 20632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "09418d8e2f3142caa5bcbe425f91be10"
format_stamp: "Formatted at 2026-08-11 03:12:12 on dist-test-slave-tm1g"
server_key: "7ba59c5d859abbc8f520405e70950aaf"
server_key_iv: "7260eb61a678a176442cc155577a22c8"
server_key_version: "encryptionkey@0"
I20260811 03:12:12.468502 20632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "09418d8e2f3142caa5bcbe425f91be10"
format_stamp: "Formatted at 2026-08-11 03:12:12 on dist-test-slave-tm1g"
server_key: "7ba59c5d859abbc8f520405e70950aaf"
server_key_iv: "7260eb61a678a176442cc155577a22c8"
server_key_version: "encryptionkey@0"
I20260811 03:12:12.473970 20632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 03:12:12.477535 20647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:12.479079 20632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 03:12:12.479269 20632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "09418d8e2f3142caa5bcbe425f91be10"
format_stamp: "Formatted at 2026-08-11 03:12:12 on dist-test-slave-tm1g"
server_key: "7ba59c5d859abbc8f520405e70950aaf"
server_key_iv: "7260eb61a678a176442cc155577a22c8"
server_key_version: "encryptionkey@0"
I20260811 03:12:12.479491 20632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:12.495460 20632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:12.499775 20632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:12.500113 20632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:12.519675 20632 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:39337
I20260811 03:12:12.519721 20703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:39337 every 8 connection(s)
I20260811 03:12:12.521157 20632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:12:12.525146 20704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:12.525348 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 20632
I20260811 03:12:12.525580 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:12:12.526453 11504 external_mini_cluster.cc:1428] Setting key 518fb677afb091e2df0a6a745abf2085
I20260811 03:12:12.539956 20704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10: Bootstrap starting.
Aug 11 03:12:12 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:12:12.543797 20704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:12.545039 20704 log.cc:824] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:12.548233 20704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10: No bootstrap required, opened a new log
I20260811 03:12:12.551585 20707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:12.528724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51282 (local address 127.11.60.62:39337)
0811 03:12:12.529535 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:12.529546 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:12.529589 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:12:12.531787 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:12.531814 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:12.532221 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:12.532569 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:12.534542 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:12.535848 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:12.537738 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:12.538540 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:12.538675 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:12.543553 (+ 4878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:12.543561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:12.543575 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:12.543612 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:12.547119 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:12.547971 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:12.547979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:12.547985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:12.548152 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:12.548614 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:12.548625 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:12.548633 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:12.549196 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:12.549555 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:12.549998 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 03:12:12.550501 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":388,"threads_started":1}
I20260811 03:12:12.556332 20704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } }
I20260811 03:12:12.556715 20704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:12.556828 20704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09418d8e2f3142caa5bcbe425f91be10, State: Initialized, Role: FOLLOWER
I20260811 03:12:12.557426 20704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } }
I20260811 03:12:12.557667 20704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:12:12.557814 20704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:12:12.557991 20704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:12.560513 20704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } }
I20260811 03:12:12.561115 20704 leader_election.cc:302] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09418d8e2f3142caa5bcbe425f91be10; no voters:
I20260811 03:12:12.561733 20704 leader_election.cc:288] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:12:12.561900 20709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:12.562733 20709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 1 LEADER]: Becoming Leader. State: Replica: 09418d8e2f3142caa5bcbe425f91be10, State: Running, Role: LEADER
I20260811 03:12:12.563304 20709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } }
I20260811 03:12:12.564095 20704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:12:12.566757 20710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09418d8e2f3142caa5bcbe425f91be10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } } }
I20260811 03:12:12.566774 20711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09418d8e2f3142caa5bcbe425f91be10. Latest consensus state: current_term: 1 leader_uuid: "09418d8e2f3142caa5bcbe425f91be10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } } }
I20260811 03:12:12.567205 20710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:12.567205 20711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:12.571681 20718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:12:12.577108 20718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:12:12.586372 20718 catalog_manager.cc:1347] Generated new cluster ID: 643c3e1c7f43406e9f2581b916659082
I20260811 03:12:12.586499 20718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:12:12.608554 20718 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:12:12.610278 20718 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:12:12.620915 20718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10: Generated new TSK 0
I20260811 03:12:12.621821 20718 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:12:12.694536 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39337
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:13.044865 20731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:13.045158 20731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:13.045364 20731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:13.052865 20731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:13.052976 20731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:13.053166 20731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:12:13.063156 20731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:13.064621 20731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:13.066200 20731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:13.080142 20738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:13.080209 20737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:13.080247 20740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:13.081075 20731 server_base.cc:1034] running on GCE node
I20260811 03:12:13.081637 20731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:13.082585 20731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:13.083782 20731 hybrid_clock.cc:648] HybridClock initialized: now 1786417933083720 us; error 52 us; skew 500 ppm
I20260811 03:12:13.084120 20731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:13 dist-test-slave-tm1g krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:13.087456 20731 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:12:13.088920 20731 webserver.cc:460] Webserver started at http://127.11.60.1:42185/ using document root <none> and password file <none>
I20260811 03:12:13.089665 20731 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:13.089798 20731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:13.090135 20731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:13.093312 20731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "b55dab612e98457f8cceccfba3a4a836"
format_stamp: "Formatted at 2026-08-11 03:12:13 on dist-test-slave-tm1g"
server_key: "df25c88e516b4836576be8e659f701bc"
server_key_iv: "69f12b8c47337238a731e9339410b05e"
server_key_version: "encryptionkey@0"
I20260811 03:12:13.094122 20731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "b55dab612e98457f8cceccfba3a4a836"
format_stamp: "Formatted at 2026-08-11 03:12:13 on dist-test-slave-tm1g"
server_key: "df25c88e516b4836576be8e659f701bc"
server_key_iv: "69f12b8c47337238a731e9339410b05e"
server_key_version: "encryptionkey@0"
I20260811 03:12:13.099880 20731 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 03:12:13.103479 20747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:13.104811 20731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:12:13.105002 20731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "b55dab612e98457f8cceccfba3a4a836"
format_stamp: "Formatted at 2026-08-11 03:12:13 on dist-test-slave-tm1g"
server_key: "df25c88e516b4836576be8e659f701bc"
server_key_iv: "69f12b8c47337238a731e9339410b05e"
server_key_version: "encryptionkey@0"
I20260811 03:12:13.105213 20731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:13.134038 20731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:13.138656 20731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:13.138967 20731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:13.139999 20731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:13.141367 20731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:13.141477 20731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:13.141636 20731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:13.141706 20731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:13.176899 20731 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:41263
I20260811 03:12:13.176949 20870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:41263 every 8 connection(s)
I20260811 03:12:13.178259 20731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:12:13.182889 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 20731
I20260811 03:12:13.183121 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:12:13.183959 11504 external_mini_cluster.cc:1428] Setting key f50fe2a47b41621c7d41c2cc73dd2b96
Aug 11 03:12:13 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:12:13.203037 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:13.181124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:57675 (local address 127.11.60.62:39337)
0811 03:12:13.184945 (+ 3821us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:13.184950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:13.184969 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:12:13.185680 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:13.185686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:13.185773 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:13.185923 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:13.188008 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:13.189205 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:13.191556 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:13.192310 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:13.192464 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:13.197024 (+ 4560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:13.197029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:13.197032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:13.197065 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:13.199766 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:13.200564 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:13.200568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:13.200570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:13.200697 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:13.201160 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:13.201163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:13.201166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:13.201430 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:13.201599 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:13.202415 (+ 816us) server_negotiation.cc:299] Negotiation successful
0811 03:12:13.202662 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3700,"thread_start_us":3577,"threads_started":1}
I20260811 03:12:13.204079 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:13.181538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:57675)
0811 03:12:13.182664 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:13.182729 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:13.185387 (+ 2658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:13.186176 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:13.186185 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:13.187015 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:13.187794 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:13.187819 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:13.189369 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:13.189376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:13.191394 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:13.191411 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:13.192557 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:13.192565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:13.192877 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:13.193682 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:13.193715 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:13.196770 (+ 3055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:13.199926 (+ 3156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:13.199936 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:13.199954 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:13.200412 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:13.200809 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:13.200813 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:13.200817 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:13.201030 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:13.201570 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:13.201578 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:13.201964 (+ 386us) client_negotiation.cc:769] Sending connection context
0811 03:12:13.202258 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 03:12:13.202866 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":372,"threads_started":1}
I20260811 03:12:13.205904 20871 heartbeater.cc:347] Connected to a master server at 127.11.60.62:39337
I20260811 03:12:13.206248 20871 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:13.207209 20871 heartbeater.cc:511] Master 127.11.60.62:39337 requested a full tablet report, sending...
I20260811 03:12:13.210074 20667 ts_manager.cc:194] Registered new tserver with Master: b55dab612e98457f8cceccfba3a4a836 (127.11.60.1:41263)
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
I20260811 03:12:13.212318 20667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:57675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:12:13.247838 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39337
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:13.597832 20877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:13.598168 20877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:13.598408 20877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:13.605721 20877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:13.605823 20877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:13.606022 20877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:12:13.616209 20877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:13.617559 20877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:13.619040 20877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:13.632589 20882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:13.632541 20885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:13.632629 20883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:13.633562 20877 server_base.cc:1034] running on GCE node
I20260811 03:12:13.634255 20877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:13.635285 20877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:13.636554 20877 hybrid_clock.cc:648] HybridClock initialized: now 1786417933636509 us; error 38 us; skew 500 ppm
I20260811 03:12:13.636888 20877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:13 dist-test-slave-tm1g krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:13.640159 20877 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:12:13.641776 20877 webserver.cc:460] Webserver started at http://127.11.60.2:37213/ using document root <none> and password file <none>
I20260811 03:12:13.642586 20877 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:13.642736 20877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:13.643069 20877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:13.645994 20877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "1737068f5f6946d5b3bea1ff48455771"
format_stamp: "Formatted at 2026-08-11 03:12:13 on dist-test-slave-tm1g"
server_key: "8fcaca8d6d7dd49adf736c9ba4f70208"
server_key_iv: "a7255735ba32c85a704e8a4f85cf9295"
server_key_version: "encryptionkey@0"
I20260811 03:12:13.646839 20877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "1737068f5f6946d5b3bea1ff48455771"
format_stamp: "Formatted at 2026-08-11 03:12:13 on dist-test-slave-tm1g"
server_key: "8fcaca8d6d7dd49adf736c9ba4f70208"
server_key_iv: "a7255735ba32c85a704e8a4f85cf9295"
server_key_version: "encryptionkey@0"
I20260811 03:12:13.652150 20877 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:12:13.655705 20894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:13.657058 20877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:12:13.657271 20877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "1737068f5f6946d5b3bea1ff48455771"
format_stamp: "Formatted at 2026-08-11 03:12:13 on dist-test-slave-tm1g"
server_key: "8fcaca8d6d7dd49adf736c9ba4f70208"
server_key_iv: "a7255735ba32c85a704e8a4f85cf9295"
server_key_version: "encryptionkey@0"
I20260811 03:12:13.657495 20877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:13.689642 20877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:13.693872 20877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:13.694226 20877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:13.695257 20877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:13.696713 20877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:13.696827 20877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:13.697005 20877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:13.697098 20877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:13.732594 20877 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:39803
I20260811 03:12:13.732686 21015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:39803 every 8 connection(s)
I20260811 03:12:13.733971 20877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:12:13.736799 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 20877
I20260811 03:12:13.737010 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:12:13.737561 11504 external_mini_cluster.cc:1428] Setting key a5e0e0a74757feb0f55946b18edd2822
Aug 11 03:12:13 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:12:13.758498 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:13.736738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:41299 (local address 127.11.60.62:39337)
0811 03:12:13.737237 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:13.737242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:13.738542 (+ 1300us) server_negotiation.cc:407] Connection header received
0811 03:12:13.741270 (+ 2728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:13.741278 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:13.741370 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:13.741498 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:13.743834 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:13.744953 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:13.747652 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:13.748403 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:13.748569 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:13.752458 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:13.752463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:13.752466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:13.752500 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:13.755230 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:13.756025 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:13.756029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:13.756031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:13.756168 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:13.756636 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:13.756646 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:13.756648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:13.756915 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:13.757091 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:13.757835 (+ 744us) server_negotiation.cc:299] Negotiation successful
0811 03:12:13.758080 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":279,"threads_started":1}
I20260811 03:12:13.759563 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:13.737371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:41299)
0811 03:12:13.738208 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:13.738274 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:13.740930 (+ 2656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:13.741789 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:13.741804 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:13.742705 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:13.743584 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:13.743614 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:13.745172 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:13.745178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:13.747470 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:13.747487 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:13.748650 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:13.748655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:13.748874 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:13.749645 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:13.749669 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:13.752193 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:13.755397 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:13.755406 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:13.755424 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:13.755869 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:13.756293 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:13.756298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:13.756302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:13.756506 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:13.757062 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:13.757070 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:13.757431 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 03:12:13.757714 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 03:12:13.758273 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":366,"threads_started":1}
I20260811 03:12:13.761258 21016 heartbeater.cc:347] Connected to a master server at 127.11.60.62:39337
I20260811 03:12:13.761600 21016 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:13.762563 21016 heartbeater.cc:511] Master 127.11.60.62:39337 requested a full tablet report, sending...
I20260811 03:12:13.764236 20667 ts_manager.cc:194] Registered new tserver with Master: 1737068f5f6946d5b3bea1ff48455771 (127.11.60.2:39803)
I20260811 03:12:13.765573 20667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:41299
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:12:13.803164 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39337
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:14.147320 21022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:14.147650 21022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:14.147840 21022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:14.155637 21022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:14.155752 21022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:14.155943 21022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:12:14.166770 21022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:14.168215 21022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:14.169688 21022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:14.187505 21030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:14.187450 21028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:14.187878 21027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:14.188627 21022 server_base.cc:1034] running on GCE node
I20260811 03:12:14.189303 21022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:14.190442 21022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:14.191704 21022 hybrid_clock.cc:648] HybridClock initialized: now 1786417934191650 us; error 50 us; skew 500 ppm
I20260811 03:12:14.192054 21022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:14 dist-test-slave-tm1g krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417934, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:14.195585 21022 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:12:14.197230 21022 webserver.cc:460] Webserver started at http://127.11.60.3:36313/ using document root <none> and password file <none>
I20260811 03:12:14.197952 21022 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:14.198056 21022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:14.198416 21022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:14.201618 21022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "464045cdf6e34e548983039d125b403b"
format_stamp: "Formatted at 2026-08-11 03:12:14 on dist-test-slave-tm1g"
server_key: "d6d6e95375e40dd555ce5b919b37f3e9"
server_key_iv: "3f08c9fc593755a48199dd1ffa765452"
server_key_version: "encryptionkey@0"
I20260811 03:12:14.202463 21022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "464045cdf6e34e548983039d125b403b"
format_stamp: "Formatted at 2026-08-11 03:12:14 on dist-test-slave-tm1g"
server_key: "d6d6e95375e40dd555ce5b919b37f3e9"
server_key_iv: "3f08c9fc593755a48199dd1ffa765452"
server_key_version: "encryptionkey@0"
I20260811 03:12:14.207820 21022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:12:14.211447 21037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:14.212800 21022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:12:14.212940 21022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "464045cdf6e34e548983039d125b403b"
format_stamp: "Formatted at 2026-08-11 03:12:14 on dist-test-slave-tm1g"
server_key: "d6d6e95375e40dd555ce5b919b37f3e9"
server_key_iv: "3f08c9fc593755a48199dd1ffa765452"
server_key_version: "encryptionkey@0"
I20260811 03:12:14.213115 21022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:14.215780 20871 heartbeater.cc:503] Master 127.11.60.62:39337 was elected leader, sending a full tablet report...
I20260811 03:12:14.227109 21022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:14.231653 21022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:14.231909 21022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:14.232832 21022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:14.234241 21022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:14.234371 21022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 03:12:14.234489 21022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:14.234539 21022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:14.269495 21022 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:44815
I20260811 03:12:14.269538 21161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:44815 every 8 connection(s)
I20260811 03:12:14.270941 21022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:12:14.271852 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 21022
I20260811 03:12:14.272055 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:12:14.272562 11504 external_mini_cluster.cc:1428] Setting key fcfcc3795fce27ff7fe471bbb11dd9c3
Aug 11 03:12:14 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417934, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:12:14.292693 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:14.273761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:48263 (local address 127.11.60.62:39337)
0811 03:12:14.274404 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:14.274411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:14.275561 (+ 1150us) server_negotiation.cc:407] Connection header received
0811 03:12:14.277614 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:14.277619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:14.277712 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:14.277820 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:14.279456 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:14.280241 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:14.281830 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:14.282250 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:14.282485 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:14.286144 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:14.286149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:14.286152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:14.286186 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:14.289687 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:14.290464 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:14.290467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:14.290470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:14.290587 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:14.291062 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:14.291066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:14.291068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:14.291373 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:14.291511 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:14.292224 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 03:12:14.292398 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":340,"threads_started":1}
I20260811 03:12:14.293677 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:14.274505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:48263)
0811 03:12:14.275406 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:14.275445 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:14.277347 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:14.278121 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:14.278129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:14.278720 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:14.279282 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:14.279298 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:14.280408 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:14.280413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:14.281681 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:14.281693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:14.282575 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:14.282582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:14.282813 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:14.283563 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:14.283586 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:14.285915 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:14.289825 (+ 3910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:14.289832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:14.289847 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:14.290360 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:14.290688 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:14.290691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:14.290694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:14.290894 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:14.291530 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:14.291536 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:14.291862 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 03:12:14.292110 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 03:12:14.292520 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":427,"threads_started":1}
I20260811 03:12:14.295173 21162 heartbeater.cc:347] Connected to a master server at 127.11.60.62:39337
I20260811 03:12:14.295540 21162 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:14.296274 21162 heartbeater.cc:511] Master 127.11.60.62:39337 requested a full tablet report, sending...
I20260811 03:12:14.297802 20667 ts_manager.cc:194] Registered new tserver with Master: 464045cdf6e34e548983039d125b403b (127.11.60.3:44815)
I20260811 03:12:14.298655 20667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:48263
I20260811 03:12:14.301460 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab.
Entry for principal oryx/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab.
WARNING: no policy specified for oryx/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab.
Entry for principal oryx/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab.
WARNING: no policy specified for oryx/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab.
Entry for principal oryx/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab.
I20260811 03:12:14.479343 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 20731
Aug 11 03:12:14 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:12:14.497403 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:14.480664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49032 (local address 127.11.60.1:41263)
0811 03:12:14.481316 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:14.481323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:14.481352 (+ 29us) server_negotiation.cc:407] Connection header received
0811 03:12:14.481548 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:14.481556 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:14.481773 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:14.481926 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:14.483749 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:14.484850 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:14.486476 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:14.487058 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:14.487163 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:14.491393 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:14.491405 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:14.491419 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:14.491460 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:14.494537 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:14.495335 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:14.495342 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:14.495344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:14.495475 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:14.495930 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:14.495933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:14.495935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:14.496297 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:14.496525 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:14.496881 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 03:12:14.497107 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":344,"threads_started":1}
I20260811 03:12:14.498780 20866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:14.588045 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 20731
I20260811 03:12:14.607609 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 20877
Aug 11 03:12:14 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:12:14.624152 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:14.608557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.11.60.2:39803)
0811 03:12:14.609110 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:14.609118 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:14.609146 (+ 28us) server_negotiation.cc:407] Connection header received
0811 03:12:14.609207 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:14.609213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:14.609429 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:14.609561 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:14.610915 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:14.611896 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:14.613210 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:14.613753 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:14.613847 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:14.618452 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:14.618463 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:14.618481 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:14.618520 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:14.621479 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:14.622146 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:14.622152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:14.622154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:14.622268 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:14.622657 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:14.622661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:14.622662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:14.622995 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:14.623224 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:14.623631 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 03:12:14.623825 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":278,"threads_started":1}
I20260811 03:12:14.625494 21010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:14.710033 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 20877
I20260811 03:12:14.730957 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 21022
Aug 11 03:12:14 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:12:14.749567 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:14.732104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58696 (local address 127.11.60.3:44815)
0811 03:12:14.732786 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:14.732794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:14.732826 (+ 32us) server_negotiation.cc:407] Connection header received
0811 03:12:14.732890 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:14.732899 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:14.733130 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:14.733266 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:14.734843 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:14.735961 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:14.737422 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:14.738043 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:14.738157 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:14.743241 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:14.743256 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:14.743273 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:14.743315 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:14.746638 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:14.747381 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:14.747389 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:14.747391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:14.747521 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:14.747918 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:14.747921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:14.747923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:14.748349 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:14.748592 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:14.748896 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 03:12:14.749119 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":293,"threads_started":1}
I20260811 03:12:14.751317 21157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:14.839349 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 21022
I20260811 03:12:14.859375 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 20632
I20260811 03:12:14.860785 20699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:14.969304 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 20632
I20260811 03:12:14.993321 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:39337
--webserver_interface=127.11.60.62
--webserver_port=34487
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:39337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:15.340008 21182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:15.340375 21182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:15.340552 21182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:15.348060 21182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:12:15.348228 21182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:15.348340 21182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:15.348449 21182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:12:15.348544 21182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:12:15.358769 21182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:39337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:39337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=34487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:15.360262 21182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:15.361711 21182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:15.374909 21187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:15.375034 21188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:15.375221 21190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:15.375878 21182 server_base.cc:1034] running on GCE node
I20260811 03:12:15.376749 21182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:15.378387 21182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:15.379607 21182 hybrid_clock.cc:648] HybridClock initialized: now 1786417935379559 us; error 29 us; skew 500 ppm
I20260811 03:12:15.380040 21182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:15 dist-test-slave-tm1g krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417935, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:15.383603 21182 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:12:15.385241 21182 webserver.cc:460] Webserver started at http://127.11.60.62:34487/ using document root <none> and password file <none>
I20260811 03:12:15.385989 21182 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:15.386107 21182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:15.392184 21182 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 03:12:15.395320 21197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:15.396957 21182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:12:15.397181 21182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "09418d8e2f3142caa5bcbe425f91be10"
format_stamp: "Formatted at 2026-08-11 03:12:12 on dist-test-slave-tm1g"
server_key: "7ba59c5d859abbc8f520405e70950aaf"
server_key_iv: "7260eb61a678a176442cc155577a22c8"
server_key_version: "encryptionkey@0"
I20260811 03:12:15.398232 21182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:15.425002 21182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:15.429768 21182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:15.430080 21182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:15.450033 21182 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:39337
I20260811 03:12:15.450071 21252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:39337 every 8 connection(s)
I20260811 03:12:15.451646 21182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:12:15.458721 21253 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:12:15.461548 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 21182
I20260811 03:12:15.462525 21253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10: Bootstrap starting.
I20260811 03:12:15.462662 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:41263
--local_ip_for_outbound_sockets=127.11.60.1
--tserver_master_addrs=127.11.60.62:39337
--webserver_port=42185
--webserver_interface=127.11.60.1
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.60.1
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:12:15.485772 21253 log.cc:824] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:15.495404 21253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:12:15.495985 21253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10: Bootstrap complete.
I20260811 03:12:15.505913 21253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } }
I20260811 03:12:15.506922 21253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09418d8e2f3142caa5bcbe425f91be10, State: Initialized, Role: FOLLOWER
I20260811 03:12:15.507547 21253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } }
I20260811 03:12:15.507813 21253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:12:15.507966 21253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:12:15.508168 21253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 1 FOLLOWER]: Advancing to term 2
I20260811 03:12:15.512195 21253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } }
I20260811 03:12:15.512735 21253 leader_election.cc:302] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09418d8e2f3142caa5bcbe425f91be10; no voters:
I20260811 03:12:15.513450 21253 leader_election.cc:288] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 03:12:15.513597 21257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 03:12:15.514902 21257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [term 2 LEADER]: Becoming Leader. State: Replica: 09418d8e2f3142caa5bcbe425f91be10, State: Running, Role: LEADER
I20260811 03:12:15.515560 21257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } }
I20260811 03:12:15.515939 21253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:12:15.518188 21259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09418d8e2f3142caa5bcbe425f91be10. Latest consensus state: current_term: 2 leader_uuid: "09418d8e2f3142caa5bcbe425f91be10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } } }
I20260811 03:12:15.518669 21259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:15.519284 21258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "09418d8e2f3142caa5bcbe425f91be10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09418d8e2f3142caa5bcbe425f91be10" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39337 } } }
I20260811 03:12:15.519663 21258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:15.528869 21263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:12:15.535032 21263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:12:15.536525 21263 catalog_manager.cc:1259] Loaded cluster ID: 643c3e1c7f43406e9f2581b916659082
I20260811 03:12:15.536696 21263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:12:15.540884 21263 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:12:15.542531 21263 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 09418d8e2f3142caa5bcbe425f91be10: Loaded TSK: 0
I20260811 03:12:15.544121 21263 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 03:12:15.843125 21255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:15.843536 21255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:15.843776 21255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:15.852721 21255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:15.852906 21255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:15.853140 21255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:12:15.864472 21255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.1.keytab
--principal=oryx/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:41263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=42185
--tserver_master_addrs=127.11.60.62:39337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:15.866012 21255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:15.867504 21255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:15.881662 21280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:15.881685 21281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:15.882504 21283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:15.883446 21255 server_base.cc:1034] running on GCE node
I20260811 03:12:15.884244 21255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:15.885445 21255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:15.886683 21255 hybrid_clock.cc:648] HybridClock initialized: now 1786417935886634 us; error 43 us; skew 500 ppm
I20260811 03:12:15.887053 21255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:15 dist-test-slave-tm1g krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417935, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:15.899410 21255 init.cc:377] Logged in from keytab as oryx/127.11.60.1@KRBTEST.COM (short username oryx)
I20260811 03:12:15.901324 21255 webserver.cc:460] Webserver started at http://127.11.60.1:42185/ using document root <none> and password file <none>
I20260811 03:12:15.902123 21255 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:15.902343 21255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:15.908281 21255 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 03:12:15.911307 21291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:15.912958 21255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:12:15.913172 21255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "b55dab612e98457f8cceccfba3a4a836"
format_stamp: "Formatted at 2026-08-11 03:12:13 on dist-test-slave-tm1g"
server_key: "df25c88e516b4836576be8e659f701bc"
server_key_iv: "69f12b8c47337238a731e9339410b05e"
server_key_version: "encryptionkey@0"
I20260811 03:12:15.914188 21255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:15.933740 21255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:15.938611 21255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:15.938936 21255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:15.940093 21255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:15.941774 21255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:15.941898 21255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:15.942092 21255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:15.942188 21255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:15.980820 21255 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:41263
I20260811 03:12:15.980876 21408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:41263 every 8 connection(s)
I20260811 03:12:15.982522 21255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:12:15.984853 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 21255
I20260811 03:12:15.985755 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:39803
--local_ip_for_outbound_sockets=127.11.60.2
--tserver_master_addrs=127.11.60.62:39337
--webserver_port=37213
--webserver_interface=127.11.60.2
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.60.2
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:12:15 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:15 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 03:12:16.006676 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:15.985767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:52713 (local address 127.11.60.62:39337)
0811 03:12:15.990882 (+ 5115us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:15.990914 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:15.990972 (+ 58us) server_negotiation.cc:407] Connection header received
0811 03:12:15.991381 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:15.991403 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:15.991896 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:15.992335 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:15.993790 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:15.996033 (+ 2243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:15.997695 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:15.998590 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:15.998736 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.005729 (+ 6993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:52713: BlockingRecv error: recv got EOF from 127.11.60.1:52713 (error 108)
Metrics: {"server-negotiator.queue_time_us":4819,"thread_start_us":4575,"threads_started":1}
I20260811 03:12:16.006711 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:15.986079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:52713)
0811 03:12:15.987198 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:15.987266 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:15.990607 (+ 3341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:15.992438 (+ 1831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:15.992446 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:15.992993 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:15.993562 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:15.993580 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:15.996224 (+ 2644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:15.996228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:15.997523 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:15.997536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:15.998841 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:15.998851 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:15.999184 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:15.999980 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:16.000014 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:16.005209 (+ 5195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":451,"threads_started":1}
W20260811 03:12:16.007807 21410 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:39337 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.60.62:39337: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:16.018647 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:16.008517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:39521)
0811 03:12:16.008743 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:16.008764 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:16.008874 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:16.009211 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:16.009219 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:16.009708 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:16.009935 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.009950 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.011207 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.011216 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.012995 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.013014 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.013694 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.013703 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.013943 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.014627 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:16.014659 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:16.018430 (+ 3771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 03:12:16.019289 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:16.008655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:39521 (local address 127.11.60.62:39337)
0811 03:12:16.008819 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:16.008822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:16.008839 (+ 17us) server_negotiation.cc:407] Connection header received
0811 03:12:16.008997 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:16.009000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:16.009059 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:16.009138 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:16.010099 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.011043 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.013146 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.013531 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.013625 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.018732 (+ 5107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:39521: BlockingRecv error: recv got EOF from 127.11.60.1:39521 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:16.028049 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:16.019495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:38335)
0811 03:12:16.019860 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:16.019881 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:16.019991 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:16.020280 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:16.020283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:16.020568 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:16.020709 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.020716 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.022108 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.022112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.023101 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.023113 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.024046 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.024050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.024191 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.024794 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:16.024810 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:16.027744 (+ 2934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240}
W20260811 03:12:16.028358 21410 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 03:12:16.028216 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:16.019817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:38335 (local address 127.11.60.62:39337)
0811 03:12:16.019992 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:16.019998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:16.020025 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:12:16.020085 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:16.020090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:16.020174 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:16.020272 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:16.020840 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.021954 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.023236 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.023901 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.024029 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.027962 (+ 3933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:38335: BlockingRecv error: recv got EOF from 127.11.60.1:38335 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 03:12:16.349776 21415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:16.350072 21415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:16.350168 21415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:16.357757 21415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:16.357882 21415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:16.357995 21415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:12:16.368036 21415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.2.keytab
--principal=oryx/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:39803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=37213
--tserver_master_addrs=127.11.60.62:39337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:16.369457 21415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:16.370935 21415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:16.384590 21421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:16.384624 21423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:16.384682 21420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:16.385666 21415 server_base.cc:1034] running on GCE node
I20260811 03:12:16.386417 21415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:16.387462 21415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:16.388855 21415 hybrid_clock.cc:648] HybridClock initialized: now 1786417936388771 us; error 57 us; skew 500 ppm
I20260811 03:12:16.389211 21415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:16.392675 21415 init.cc:377] Logged in from keytab as oryx/127.11.60.2@KRBTEST.COM (short username oryx)
I20260811 03:12:16.394279 21415 webserver.cc:460] Webserver started at http://127.11.60.2:37213/ using document root <none> and password file <none>
I20260811 03:12:16.395144 21415 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:16.395303 21415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:16.401238 21415 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 03:12:16.404614 21432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:16.406124 21415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:12:16.406414 21415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "1737068f5f6946d5b3bea1ff48455771"
format_stamp: "Formatted at 2026-08-11 03:12:13 on dist-test-slave-tm1g"
server_key: "8fcaca8d6d7dd49adf736c9ba4f70208"
server_key_iv: "a7255735ba32c85a704e8a4f85cf9295"
server_key_version: "encryptionkey@0"
I20260811 03:12:16.407418 21415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:16.443636 21415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:16.448467 21415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:16.448768 21415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:16.450062 21415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:16.451519 21415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:16.451620 21415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:16.451784 21415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:16.451884 21415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:16.487010 21415 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:39803
I20260811 03:12:16.487211 21552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:39803 every 8 connection(s)
I20260811 03:12:16.488554 21415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:12:16.490984 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 21415
I20260811 03:12:16.491806 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:44815
--local_ip_for_outbound_sockets=127.11.60.3
--tserver_master_addrs=127.11.60.62:39337
--webserver_port=36313
--webserver_interface=127.11.60.3
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.60.3
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 03:12:16.506487 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:16.491502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:55147 (local address 127.11.60.62:39337)
0811 03:12:16.491717 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:16.491722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:16.493216 (+ 1494us) server_negotiation.cc:407] Connection header received
0811 03:12:16.495215 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:16.495221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:16.495320 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:16.495443 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:16.497107 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.497841 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.499585 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.500045 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.500202 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.506185 (+ 5983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:55147: BlockingRecv error: recv got EOF from 127.11.60.2:55147 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 03:12:16.507266 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:16.492179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:55147)
0811 03:12:16.493019 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:16.493069 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:16.494915 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:16.495703 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:16.495711 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:16.496258 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:16.496892 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.496911 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.498044 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.498048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.499425 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.499437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.500299 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.500307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.500563 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.501365 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:16.501401 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:16.505975 (+ 4574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":352,"threads_started":1}
W20260811 03:12:16.508028 21553 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:39337 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.60.62:39337: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:16.519052 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:16.508743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:37023)
0811 03:12:16.509036 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:16.509057 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:16.509168 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:16.509508 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:16.509515 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:16.510000 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:16.510250 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.510266 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.511378 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.511386 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.513160 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.513178 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.513925 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.513933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.514170 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.514910 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:16.514941 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:16.518710 (+ 3769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149}
W20260811 03:12:16.519186 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:16.508954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:37023 (local address 127.11.60.62:39337)
0811 03:12:16.509153 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:16.509157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:16.509175 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:12:16.509290 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:16.509293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:16.509355 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:16.509442 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:16.510427 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.511229 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.513318 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.513755 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.513852 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.518943 (+ 5091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:37023: BlockingRecv error: recv got EOF from 127.11.60.2:37023 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:16.528013 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:16.519960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:37699)
0811 03:12:16.520119 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:16.520137 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:16.520267 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:16.520652 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:16.520657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:16.520974 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:16.521127 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.521134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.522181 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.522187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.523219 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.523230 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.523954 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.523963 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.524159 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.524801 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:16.524818 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:16.527694 (+ 2876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 03:12:16.528405 21553 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 03:12:16.528595 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:16.520168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:37699 (local address 127.11.60.62:39337)
0811 03:12:16.520374 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:16.520379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:16.520395 (+ 16us) server_negotiation.cc:407] Connection header received
0811 03:12:16.520448 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:16.520451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:16.520516 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:16.520622 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:16.521280 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.522038 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.523369 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.523798 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.523941 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.527904 (+ 3963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:37699: BlockingRecv error: recv got EOF from 127.11.60.2:37699 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 03:12:16.851315 21556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:16.851665 21556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:16.851850 21556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:16.859423 21556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:16.859588 21556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:16.859784 21556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:12:16.869791 21556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/krb5kdc/oryx_127.11.60.3.keytab
--principal=oryx/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:44815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=36313
--tserver_master_addrs=127.11.60.62:39337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:16.871261 21556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:16.872637 21556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:16.886606 21562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:16.886675 21561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:16.886693 21565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:16.892151 21556 server_base.cc:1034] running on GCE node
I20260811 03:12:16.892867 21556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:16.893970 21556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:16.895262 21556 hybrid_clock.cc:648] HybridClock initialized: now 1786417936895218 us; error 53 us; skew 500 ppm
I20260811 03:12:16.895632 21556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:16.899071 21556 init.cc:377] Logged in from keytab as oryx/127.11.60.3@KRBTEST.COM (short username oryx)
I20260811 03:12:16.900872 21556 webserver.cc:460] Webserver started at http://127.11.60.3:36313/ using document root <none> and password file <none>
I20260811 03:12:16.901690 21556 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:16.901863 21556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:16.907624 21556 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 03:12:16.910517 21573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:16.912081 21556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:12:16.912271 21556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "464045cdf6e34e548983039d125b403b"
format_stamp: "Formatted at 2026-08-11 03:12:14 on dist-test-slave-tm1g"
server_key: "d6d6e95375e40dd555ce5b919b37f3e9"
server_key_iv: "3f08c9fc593755a48199dd1ffa765452"
server_key_version: "encryptionkey@0"
I20260811 03:12:16.913295 21556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:16.940608 21556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:16.944913 21556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:16.945286 21556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:16.946528 21556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:16.947926 21556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:16.948063 21556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:16.948208 21556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:16.948312 21556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:16.985616 21556 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:44815
I20260811 03:12:16.985715 21694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:44815 every 8 connection(s)
I20260811 03:12:16.987164 21556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:12:16.994445 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 21556
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:16 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 03:12:17.004503 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:16.989829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:55069 (local address 127.11.60.62:39337)
0811 03:12:16.990035 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:16.990040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:16.991318 (+ 1278us) server_negotiation.cc:407] Connection header received
0811 03:12:16.993202 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:16.993208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:16.993311 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:16.993433 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:16.995153 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.996327 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.997921 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.998438 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.998593 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:17.004195 (+ 5602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:55069: BlockingRecv error: recv got EOF from 127.11.60.3:55069 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:12:17.005303 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:16.990409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:55069)
0811 03:12:16.991117 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:16.991173 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:16.992914 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:16.993673 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:16.993683 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:16.994216 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:16.994956 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.994980 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.996507 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.996512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.997763 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:16.997775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.998688 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:16.998696 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:16.998937 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:16.999666 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:16.999696 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:17.004007 (+ 4311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":276,"threads_started":1}
W20260811 03:12:17.006026 21695 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:39337 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.60.62:39337: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
I20260811 03:12:17.012274 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:16.995625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51286 (local address 127.11.60.62:39337)
0811 03:12:16.996401 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:16.996408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:16.996429 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:12:16.996484 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:16.996489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:16.996565 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:16.996683 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:16.998250 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:16.999142 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.001014 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.001426 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.001579 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:17.004524 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:17.004532 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:17.004551 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:17.004628 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:17.008609 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:17.009327 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:17.009336 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:17.009347 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:17.009555 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:17.009961 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:17.009966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:17.009973 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:17.010548 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:17.010888 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:17.011401 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 03:12:17.011799 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":526,"threads_started":1}
Aug 11 03:12:17 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:17 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:17.016340 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:17.006730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:52101)
0811 03:12:17.006957 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:17.006978 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:17.007081 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:17.007354 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:17.007357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:17.007661 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:17.007791 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:17.007799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.008843 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.008850 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:17.009760 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:17.009775 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.010630 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.010636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:17.010835 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:17.011399 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:17.011420 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:17.016049 (+ 4629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 03:12:17.016464 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:17.006910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:52101 (local address 127.11.60.62:39337)
0811 03:12:17.007089 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:17.007095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:17.007115 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:12:17.007170 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:17.007174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:17.007248 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:17.007350 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:17.007946 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.008695 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.009909 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.010469 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.010613 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:17.016243 (+ 5630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:52101: BlockingRecv error: recv got EOF from 127.11.60.3:52101 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 03:12:17 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:17 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:17.024585 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:17.017023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:56579)
0811 03:12:17.017181 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:17.017206 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:17.017311 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:17.017600 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:17.017603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:17.017918 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:17.018116 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:17.018126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.019343 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.019347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:17.020347 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:17.020359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.021027 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.021031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:17.021151 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:17.021681 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:17.021697 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:17.024357 (+ 2660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 03:12:17.024883 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:17.017117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:56579 (local address 127.11.60.62:39337)
0811 03:12:17.017342 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:17.017347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:17.017363 (+ 16us) server_negotiation.cc:407] Connection header received
0811 03:12:17.017401 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:17.017404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:17.017455 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:17.017529 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:17.018332 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.019179 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.020497 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.020898 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.021002 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:17.024589 (+ 3587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:56579: BlockingRecv error: recv got EOF from 127.11.60.3:56579 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
W20260811 03:12:17.024917 21695 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 03:12:17 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:17 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:17.038314 21700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:17.028985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:60861)
0811 03:12:17.029578 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:17.029595 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:17.029763 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:17.030127 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:17.030132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:17.030744 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:17.030944 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:17.030955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.032046 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.032052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:17.033038 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:17.033051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.033870 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.033877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:17.034045 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:17.034802 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:17.034821 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:17.037961 (+ 3140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":319,"threads_started":1}
W20260811 03:12:17.038501 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:17.029145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:60861 (local address 127.11.60.62:39337)
0811 03:12:17.029380 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:17.029389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:17.029688 (+ 299us) server_negotiation.cc:407] Connection header received
0811 03:12:17.029915 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:17.029925 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:17.029986 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:17.030092 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:17.031098 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.031872 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.033201 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.033663 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.033803 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:17.038220 (+ 4417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:60861: BlockingRecv error: recv got EOF from 127.11.60.1:60861 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 03:12:17 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:17 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:17.537724 21701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:17.529072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:58647)
0811 03:12:17.529676 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:17.529694 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:17.529814 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:17.530133 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:17.530136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:17.530704 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:17.530915 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:17.530924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.531971 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.531975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:17.532947 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:17.532959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.533728 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.533732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:17.533859 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:17.534655 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:17.534674 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:17.537489 (+ 2815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":345,"threads_started":1}
W20260811 03:12:17.537997 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:17.529137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:58647 (local address 127.11.60.62:39337)
0811 03:12:17.529312 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:17.529316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:17.529773 (+ 457us) server_negotiation.cc:407] Connection header received
0811 03:12:17.529910 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:17.529914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:17.529994 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:17.530075 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:17.531080 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.531808 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.533091 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:17.533550 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:17.533649 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:17.537697 (+ 4048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:58647: BlockingRecv error: recv got EOF from 127.11.60.2:58647 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 03:12:18 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:18 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:18.034159 21702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:18.025577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:43667)
0811 03:12:18.026138 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:18.026159 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:18.026276 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:18.026675 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:18.026679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:18.027175 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:18.027377 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:18.027386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.028505 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.028510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:18.029513 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:18.029524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.030233 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.030238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:18.030456 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:18.031085 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:18.031102 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:18.033932 (+ 2830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":299,"threads_started":1}
W20260811 03:12:18.034562 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:18.025621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:43667 (local address 127.11.60.62:39337)
0811 03:12:18.025803 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:18.025807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:18.026238 (+ 431us) server_negotiation.cc:407] Connection header received
0811 03:12:18.026456 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:18.026459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:18.026542 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:18.026626 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:18.027544 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.028307 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.029655 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.030059 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.030154 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:18.034244 (+ 4090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:43667: BlockingRecv error: recv got EOF from 127.11.60.3:43667 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 03:12:18 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:18 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:18.048127 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:18.039272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:56495)
0811 03:12:18.039823 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:18.039839 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:18.039953 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:18.040311 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:18.040314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:18.040684 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:18.040836 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:18.040844 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.041868 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.041872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:18.042865 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:18.042878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.043619 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.043624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:18.043740 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:18.044312 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:18.044330 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:18.047725 (+ 3395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":306,"threads_started":1}
W20260811 03:12:18.048311 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:18.039352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:56495 (local address 127.11.60.62:39337)
0811 03:12:18.039562 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:18.039566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:18.039977 (+ 411us) server_negotiation.cc:407] Connection header received
0811 03:12:18.040105 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:18.040108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:18.040178 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:18.040261 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:18.041008 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.041726 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.043021 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.043461 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.043562 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:18.047991 (+ 4429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:56495: BlockingRecv error: recv got EOF from 127.11.60.1:56495 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 11 03:12:18 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:18 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:18.548100 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:18.538987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:52085)
0811 03:12:18.539646 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:18.539666 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:18.539800 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:18.540171 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:18.540174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:18.540729 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:18.540885 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:18.540893 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.542357 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.542363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:18.543372 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:18.543384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.544121 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.544125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:18.544248 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:18.544882 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:18.544899 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:18.547843 (+ 2944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":345,"threads_started":1}
W20260811 03:12:18.548306 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:18.539118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:52085 (local address 127.11.60.62:39337)
0811 03:12:18.539330 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:18.539334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:18.539790 (+ 456us) server_negotiation.cc:407] Connection header received
0811 03:12:18.539923 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:18.539927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:18.540009 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:18.540163 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:18.541029 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.542157 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.543523 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:18.543972 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:18.544083 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:18.548082 (+ 3999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:52085: BlockingRecv error: recv got EOF from 127.11.60.2:52085 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 03:12:19 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:19 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:19.044312 21705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:19.035259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:52297)
0811 03:12:19.035897 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:19.035921 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:19.036097 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:19.036445 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:19.036450 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:19.036909 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:19.037075 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:19.037084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.038404 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.038416 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:19.039334 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:19.039346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.040117 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.040124 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:19.040300 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:19.040978 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:19.040997 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:19.043935 (+ 2938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":381,"threads_started":1}
W20260811 03:12:19.044409 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:19.035513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:52297 (local address 127.11.60.62:39337)
0811 03:12:19.035768 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:19.035776 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:19.036014 (+ 238us) server_negotiation.cc:407] Connection header received
0811 03:12:19.036237 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:19.036242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:19.036304 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:19.036414 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:19.037236 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.038225 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.039488 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.039925 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.040099 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:19.044142 (+ 4043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:52297: BlockingRecv error: recv got EOF from 127.11.60.3:52297 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 11 03:12:19 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:19 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:19.057772 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:19.049221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:49877)
0811 03:12:19.049817 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:19.049838 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:19.049987 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:19.050415 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:19.050420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:19.050911 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:19.051147 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:19.051162 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.052181 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.052185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:19.053084 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:19.053095 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.053726 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.053731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:19.053858 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:19.054531 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:19.054550 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:19.057501 (+ 2951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":315,"threads_started":1}
W20260811 03:12:19.057986 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:19.049259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:49877 (local address 127.11.60.62:39337)
0811 03:12:19.049486 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:19.049493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:19.049957 (+ 464us) server_negotiation.cc:407] Connection header received
0811 03:12:19.050101 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:19.050108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:19.050215 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:19.050357 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:19.051306 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.052042 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.053212 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.053598 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.053691 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:19.057709 (+ 4018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:49877: BlockingRecv error: recv got EOF from 127.11.60.1:49877 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 03:12:19 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:19 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:19.557870 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:19.549052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:57985)
0811 03:12:19.549576 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:19.549594 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:19.549711 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:19.550016 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:19.550019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:19.550459 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:19.550674 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:19.550684 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.551744 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.551747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:19.552746 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:19.552758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.553467 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.553471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:19.553618 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:19.554249 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:19.554267 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:19.557462 (+ 3195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":258,"threads_started":1}
W20260811 03:12:19.557924 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:19.549158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:57985 (local address 127.11.60.62:39337)
0811 03:12:19.549416 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:19.549423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:19.549676 (+ 253us) server_negotiation.cc:407] Connection header received
0811 03:12:19.549813 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:19.549816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:19.549882 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:19.549983 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:19.550823 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.551628 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.552900 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:19.553323 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:19.553443 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:19.557670 (+ 4227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:57985: BlockingRecv error: recv got EOF from 127.11.60.2:57985 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 11 03:12:20 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:20 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:20.053730 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:20.045394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:55555)
0811 03:12:20.045982 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:20.046007 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:20.046132 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:20.046439 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:20.046443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:20.046878 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:20.047074 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:20.047083 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.048147 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.048151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:20.049064 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:20.049076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.049756 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.049760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:20.049882 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:20.050557 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:20.050576 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:20.053445 (+ 2869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":308,"threads_started":1}
W20260811 03:12:20.053884 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:20.045598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:55555 (local address 127.11.60.62:39337)
0811 03:12:20.045852 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:20.045859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:20.046094 (+ 235us) server_negotiation.cc:407] Connection header received
0811 03:12:20.046224 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:20.046227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:20.046293 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:20.046390 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:20.047235 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.048000 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.049200 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.049625 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.049722 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:20.053673 (+ 3951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:55555: BlockingRecv error: recv got EOF from 127.11.60.3:55555 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 03:12:20 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:20 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:20.067042 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:20.058739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:40027)
0811 03:12:20.059249 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:20.059266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:20.059376 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:20.059661 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:20.059664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:20.060020 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:20.060184 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:20.060192 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.061209 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.061213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:20.062217 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:20.062229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.062924 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.062928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:20.063069 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:20.063635 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:20.063652 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:20.066678 (+ 3026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":277,"threads_started":1}
W20260811 03:12:20.067276 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:20.058787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:40027 (local address 127.11.60.62:39337)
0811 03:12:20.058973 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:20.058976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:20.059335 (+ 359us) server_negotiation.cc:407] Connection header received
0811 03:12:20.059467 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:20.059471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:20.059531 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:20.059612 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:20.060302 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.061062 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.062388 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.062801 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.062892 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:20.066943 (+ 4051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:40027: BlockingRecv error: recv got EOF from 127.11.60.1:40027 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 03:12:20 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:20 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:20.568338 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:20.559027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:48797)
0811 03:12:20.559616 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:20.559634 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:20.559751 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:20.560107 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:20.560111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:20.560508 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:20.560663 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:20.560671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.561954 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.561968 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:20.563018 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:20.563033 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.563791 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.563796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:20.563933 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:20.564597 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:20.564620 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:20.567976 (+ 3356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":341,"threads_started":1}
W20260811 03:12:20.568495 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:20.559070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:48797 (local address 127.11.60.62:39337)
0811 03:12:20.559266 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:20.559270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:20.559745 (+ 475us) server_negotiation.cc:407] Connection header received
0811 03:12:20.559849 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:20.559853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:20.559935 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:20.560031 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:20.560828 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.561707 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.563184 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:20.563648 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:20.563762 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:20.568197 (+ 4435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:48797: BlockingRecv error: recv got EOF from 127.11.60.2:48797 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 03:12:21 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:21 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:21.067839 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:21.054822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:47581)
0811 03:12:21.055423 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:21.055444 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:21.055624 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:21.056039 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:21.056045 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:21.056507 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:21.056668 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:21.056678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.057831 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.057841 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:21.058968 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:21.058981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.059792 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.059799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:21.060010 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:21.064265 (+ 4255us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:21.064287 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:21.067492 (+ 3205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":321,"threads_started":1}
W20260811 03:12:21.068017 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:21.054930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:47581 (local address 127.11.60.62:39337)
0811 03:12:21.055202 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:21.055211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:21.055549 (+ 338us) server_negotiation.cc:407] Connection header received
0811 03:12:21.055768 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:21.055774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:21.055875 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:21.055997 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:21.056844 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.057685 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.059132 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.059621 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.059779 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:21.067749 (+ 7970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:47581: BlockingRecv error: recv got EOF from 127.11.60.3:47581 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 11 03:12:21 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:21 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:21.077097 21713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:21.068266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:34621)
0811 03:12:21.068886 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:21.068902 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:21.069017 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:21.069397 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:21.069400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:21.069834 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:21.070002 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:21.070011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.071099 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.071104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:21.072081 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:21.072094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.072807 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.072812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:21.072969 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:21.073635 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:21.073654 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:21.076676 (+ 3022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":354,"threads_started":1}
W20260811 03:12:21.077147 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:21.068451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:34621 (local address 127.11.60.62:39337)
0811 03:12:21.068669 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:21.068674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:21.068982 (+ 308us) server_negotiation.cc:407] Connection header received
0811 03:12:21.069186 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:21.069190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:21.069253 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:21.069339 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:21.070150 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.070949 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.072209 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.072638 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.072737 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:21.076899 (+ 4162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:34621: BlockingRecv error: recv got EOF from 127.11.60.1:34621 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 03:12:21 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:21 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:21.578219 21714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:21.569393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:48025)
0811 03:12:21.569944 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:21.569970 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:21.570124 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:21.570529 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:21.570533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:21.570973 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:21.571165 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:21.571174 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.572219 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.572226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:21.573170 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:21.573180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.573903 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.573909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:21.574087 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:21.574740 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:21.574757 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:21.577951 (+ 3194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":251,"threads_started":1}
W20260811 03:12:21.578545 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:21.569671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:48025 (local address 127.11.60.62:39337)
0811 03:12:21.569901 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:21.569908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:21.570054 (+ 146us) server_negotiation.cc:407] Connection header received
0811 03:12:21.570256 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:21.570261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:21.570358 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:21.570492 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:21.571321 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.572069 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.573315 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:21.573753 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:21.573888 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:21.578237 (+ 4349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:48025: BlockingRecv error: recv got EOF from 127.11.60.2:48025 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 11 03:12:22 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:22 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:22.082953 21715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:22.069055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:52017)
0811 03:12:22.071487 (+ 2432us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:22.071536 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:22.071690 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:22.072321 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:22.072328 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:22.073017 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:22.073305 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:22.073323 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.074994 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.075000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:22.076840 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:22.076857 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.077973 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.077982 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:22.078168 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:22.078894 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:22.078922 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:22.082578 (+ 3656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2287,"thread_start_us":2120,"threads_started":1}
W20260811 03:12:22.083065 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:22.069159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:52017 (local address 127.11.60.62:39337)
0811 03:12:22.069395 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:22.069404 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:22.071639 (+ 2235us) server_negotiation.cc:407] Connection header received
0811 03:12:22.071887 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:22.071892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:22.071998 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:22.072127 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:22.073532 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.074783 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.077043 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.077788 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.077928 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:22.082809 (+ 4881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:52017: BlockingRecv error: recv got EOF from 127.11.60.3:52017 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 03:12:22 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:22 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:22.095157 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:22.078146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:53143)
0811 03:12:22.078895 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:22.078917 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:22.079051 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:22.084246 (+ 5195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:22.084252 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:22.085217 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:22.085405 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:22.085416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.086852 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.086858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:22.088634 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:22.088672 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.089613 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.089645 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:22.089840 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:22.090801 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:22.090853 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:22.094787 (+ 3934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":801,"threads_started":1}
W20260811 03:12:22.095428 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:22.078762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:53143 (local address 127.11.60.62:39337)
0811 03:12:22.083287 (+ 4525us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:22.083292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:22.083313 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:12:22.083384 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:22.083391 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:22.083501 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:22.083608 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:22.085568 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.086680 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.088811 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.089445 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.089599 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:22.095106 (+ 5507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:53143: BlockingRecv error: recv got EOF from 127.11.60.1:53143 (error 108)
Metrics: {"server-negotiator.queue_time_us":4415,"thread_start_us":12486,"threads_started":1}
Aug 11 03:12:22 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:22 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:22.588316 21718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:22.579307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:40923)
0811 03:12:22.579971 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:22.579987 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:22.580175 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:22.580533 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:22.580538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:22.581004 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:22.581172 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:22.581181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.582249 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.582266 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:22.583243 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:22.583254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.584003 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.584010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:22.584199 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:22.584946 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:22.584964 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:22.588092 (+ 3128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":386,"threads_started":1}
W20260811 03:12:22.588642 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:22.579419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:40923 (local address 127.11.60.62:39337)
0811 03:12:22.579696 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:22.579705 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:22.580102 (+ 397us) server_negotiation.cc:407] Connection header received
0811 03:12:22.580314 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:22.580323 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:22.580387 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:22.580496 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:22.581350 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.582102 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.583400 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:22.583830 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:22.583966 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:22.588325 (+ 4359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:40923: BlockingRecv error: recv got EOF from 127.11.60.2:40923 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 03:12:23 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:23 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:23.100924 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:23.091441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:57441)
0811 03:12:23.092043 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:23.092064 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:23.092183 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:23.092552 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:23.092560 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:23.092933 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:23.093105 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:23.093114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.094139 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.094142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:23.095229 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:23.095241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.096261 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.096270 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:23.096487 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:23.097243 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:23.097270 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:23.100584 (+ 3314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":353,"threads_started":1}
W20260811 03:12:23.101192 21720 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:23.091540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:57441 (local address 127.11.60.62:39337)
0811 03:12:23.092036 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:23.092040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:23.092139 (+ 99us) server_negotiation.cc:407] Connection header received
0811 03:12:23.092320 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:23.092324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:23.092387 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:23.092484 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:23.093237 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.094011 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.095375 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.095852 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.095993 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:23.100799 (+ 4806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:57441: BlockingRecv error: recv got EOF from 127.11.60.3:57441 (error 108)
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":266,"threads_started":1}
Aug 11 03:12:23 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:23 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:23.105666 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:23.096298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:35237)
0811 03:12:23.097020 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:23.097042 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:23.097158 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:23.097578 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:23.097581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:23.097962 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:23.098153 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:23.098161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.099509 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.099514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:23.100479 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:23.100491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.101598 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.101604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:23.101789 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:23.102370 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:23.102388 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:23.105293 (+ 2905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":281,"threads_started":1}
W20260811 03:12:23.105922 21722 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:23.096412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:35237 (local address 127.11.60.62:39337)
0811 03:12:23.097265 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:23.097271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:23.097288 (+ 17us) server_negotiation.cc:407] Connection header received
0811 03:12:23.097339 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:23.097346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:23.097427 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:23.097547 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:23.098351 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.099299 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.100620 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.101433 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.101568 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:23.105605 (+ 4037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:35237: BlockingRecv error: recv got EOF from 127.11.60.1:35237 (error 108)
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":562,"threads_started":1}
Aug 11 03:12:23 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:23 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:23.599092 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:23.589542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:48173)
0811 03:12:23.590193 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:23.590210 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:23.590419 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:23.590784 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:23.590789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:23.591301 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:23.591485 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:23.591494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.592585 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.592592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:23.593652 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:23.593665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.594464 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.594471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:23.594650 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:23.595285 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:23.595304 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:23.598680 (+ 3376us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":301,"threads_started":1}
W20260811 03:12:23.599221 21722 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:23.589661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:48173 (local address 127.11.60.62:39337)
0811 03:12:23.589949 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:23.589956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:23.590337 (+ 381us) server_negotiation.cc:407] Connection header received
0811 03:12:23.590557 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:23.590562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:23.590638 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:23.590757 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:23.591655 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.592413 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.593807 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:23.594267 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:23.594444 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:23.598950 (+ 4506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:48173: BlockingRecv error: recv got EOF from 127.11.60.2:48173 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 11 03:12:24 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:24 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:24.112252 21725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:24.102090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:49259)
0811 03:12:24.102747 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:24.102767 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:24.102937 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:24.103319 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:24.103323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:24.103799 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:24.103981 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:24.103990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.105407 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.105413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:24.106756 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:24.106770 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.107825 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.107831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:24.108016 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:24.108626 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:24.108643 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:24.111825 (+ 3182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":314,"threads_started":1}
W20260811 03:12:24.112293 21726 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:24.102131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:49259 (local address 127.11.60.62:39337)
0811 03:12:24.102749 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:24.102754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:24.102901 (+ 147us) server_negotiation.cc:407] Connection header received
0811 03:12:24.103078 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:24.103082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:24.103176 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:24.103270 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:24.104128 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.105231 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.106908 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.107670 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.107805 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:24.112020 (+ 4215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:49259: BlockingRecv error: recv got EOF from 127.11.60.3:49259 (error 108)
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":395,"threads_started":1}
Aug 11 03:12:24 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:24 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:24.115828 21727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:24.106636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:58621)
0811 03:12:24.107319 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:24.107340 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:24.107461 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:24.107968 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:24.107974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:24.108589 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:24.108756 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:24.108767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.109854 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.109858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:24.110937 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:24.110961 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.111827 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.111833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:24.112051 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:24.112679 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:24.112704 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:24.115593 (+ 2889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":393,"threads_started":1}
W20260811 03:12:24.116174 21728 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:24.106846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:58621 (local address 127.11.60.62:39337)
0811 03:12:24.107698 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:24.107702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:24.107716 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:12:24.107753 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:24.107756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:24.107808 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:24.107926 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:24.108925 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.109701 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.111107 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.111622 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.111761 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:24.115838 (+ 4077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:58621: BlockingRecv error: recv got EOF from 127.11.60.1:58621 (error 108)
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":574,"threads_started":1}
Aug 11 03:12:24 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:24 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:24.609364 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:24.600165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:53601)
0811 03:12:24.600785 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:24.600801 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:24.600969 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:24.601329 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:24.601335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:24.601758 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:24.602003 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:24.602013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.603460 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.603466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:24.604412 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:24.604423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.605173 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.605180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:24.605360 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:24.606040 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:24.606058 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:24.608999 (+ 2941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":369,"threads_started":1}
W20260811 03:12:24.609464 21728 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:24.600332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:53601 (local address 127.11.60.62:39337)
0811 03:12:24.600591 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:24.600598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:24.600893 (+ 295us) server_negotiation.cc:407] Connection header received
0811 03:12:24.601100 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:24.601106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:24.601183 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:24.601287 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:24.602149 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.603310 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.604562 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:24.604979 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:24.605109 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:24.609233 (+ 4124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:53601: BlockingRecv error: recv got EOF from 127.11.60.2:53601 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 03:12:25 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:25 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:25.123900 21730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:25.113449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:43245)
0811 03:12:25.113962 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:25.113983 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:25.114090 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:25.114545 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:25.114550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:25.114948 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:25.115111 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:25.115119 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.116207 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.116212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:25.117616 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:25.117635 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.118973 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.118984 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:25.119161 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:25.119905 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:25.119930 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:25.123563 (+ 3633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":260,"threads_started":1}
W20260811 03:12:25.124130 21731 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:25.113524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:43245 (local address 127.11.60.62:39337)
0811 03:12:25.114253 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:25.114257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:25.114273 (+ 16us) server_negotiation.cc:407] Connection header received
0811 03:12:25.114334 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:25.114337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:25.114409 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:25.114496 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:25.115245 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.116064 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.117912 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.118660 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.118780 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:25.123795 (+ 5015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:43245: BlockingRecv error: recv got EOF from 127.11.60.3:43245 (error 108)
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":535,"threads_started":1}
Aug 11 03:12:25 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:25 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:25.127450 21732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:25.117280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:52207)
0811 03:12:25.118124 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:25.118142 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:25.118266 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:25.118690 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:25.118695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:25.119535 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:25.119762 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:25.119776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.121293 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.121299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:25.122776 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:25.122789 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.123529 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.123535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:25.123732 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:25.124318 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:25.124341 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:25.127196 (+ 2855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":499,"threads_started":1}
W20260811 03:12:25.127712 21733 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:25.117301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:52207 (local address 127.11.60.62:39337)
0811 03:12:25.117961 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:25.117967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:25.118222 (+ 255us) server_negotiation.cc:407] Connection header received
0811 03:12:25.118450 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:25.118455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:25.118539 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:25.118639 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:25.119918 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.121132 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.122929 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.123367 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.123504 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:25.127428 (+ 3924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:52207: BlockingRecv error: recv got EOF from 127.11.60.1:52207 (error 108)
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":251,"threads_started":1}
Aug 11 03:12:25 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:25 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:25.619863 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:25.610738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:37263)
0811 03:12:25.611312 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:25.611329 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:25.611512 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:25.611823 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:25.611826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:25.612262 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:25.612518 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:25.612530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.613762 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.613767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:25.614870 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:25.614882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.615566 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.615571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:25.615693 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:25.616353 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:25.616371 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:25.619603 (+ 3232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":317,"threads_started":1}
W20260811 03:12:25.620127 21733 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:25.610777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:37263 (local address 127.11.60.62:39337)
0811 03:12:25.610953 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:25.610958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:25.611507 (+ 549us) server_negotiation.cc:407] Connection header received
0811 03:12:25.611609 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:25.611613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:25.611681 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:25.611773 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:25.612686 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.613601 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.615003 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:25.615415 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:25.615539 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:25.619843 (+ 4304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:37263: BlockingRecv error: recv got EOF from 127.11.60.2:37263 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 03:12:26 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:26 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:26 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:26 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:26.140146 21736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:26.125002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:43341)
0811 03:12:26.125816 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:26.125837 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:26.125963 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:26.126286 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:26.126290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:26.126780 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:26.126957 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:26.126967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.128029 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.128035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:26.129529 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:26.129543 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.130373 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.130380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:26.130606 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:26.136468 (+ 5862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:26.136488 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:26.139768 (+ 3280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":568,"threads_started":1}
W20260811 03:12:26.140275 21735 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:26.125033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:43341 (local address 127.11.60.62:39337)
0811 03:12:26.125456 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:26.125461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:26.125917 (+ 456us) server_negotiation.cc:407] Connection header received
0811 03:12:26.126062 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:26.126066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:26.126152 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:26.126241 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:26.127100 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.127874 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.129688 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.130172 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.130319 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:26.139993 (+ 9674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:43341: BlockingRecv error: recv got EOF from 127.11.60.3:43341 (error 108)
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":216,"threads_started":1}
I20260811 03:12:26.140305 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:26.128413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:43479)
0811 03:12:26.128927 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:26.128957 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:26.129050 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:26.129696 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:26.129703 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:26.130109 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:26.130254 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:26.130264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.131561 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.131568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:26.132893 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:26.132907 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.133607 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.133611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:26.133735 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:26.136522 (+ 2787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:26.136540 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:26.139964 (+ 3424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":289,"threads_started":1}
W20260811 03:12:26.140578 21738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:26.128490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:43479 (local address 127.11.60.62:39337)
0811 03:12:26.129404 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:26.129408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:26.129420 (+ 12us) server_negotiation.cc:407] Connection header received
0811 03:12:26.129471 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:26.129474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:26.129526 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:26.129643 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:26.130445 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.131404 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.133025 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.133471 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.133566 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:26.140191 (+ 6625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:43479: BlockingRecv error: recv got EOF from 127.11.60.1:43479 (error 108)
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":719,"threads_started":1}
Aug 11 03:12:26 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:26 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 03:12:26.629833 21738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:26.620943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:49683 (local address 127.11.60.62:39337)
0811 03:12:26.621119 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:26.621123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:26.621482 (+ 359us) server_negotiation.cc:407] Connection header received
0811 03:12:26.621621 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:26.621624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:26.621686 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:26.621770 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:26.622586 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.623499 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.625088 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.625674 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.625775 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:26.629640 (+ 3865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:49683: BlockingRecv error: recv got EOF from 127.11.60.2:49683 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 03:12:26.629961 21739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:26.620813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:49683)
0811 03:12:26.621390 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:26.621406 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:26.621521 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:26.621820 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:26.621823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:26.622217 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:26.622441 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:26.622452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.623688 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.623694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:26.624908 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:26.624922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:26.625829 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:26.625833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:26.625968 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:26.626618 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:26.626636 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:26.629591 (+ 2955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":342,"threads_started":1}
Aug 11 03:12:27 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:27 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:27 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:27 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:27.158506 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:27.141352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:33601)
0811 03:12:27.148383 (+ 7031us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:27.148413 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:27.148557 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:27.149466 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:27.149471 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:27.150067 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:27.150414 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:27.150431 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.152316 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.152320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:27.153253 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:27.153266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.154097 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.154106 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:27.154298 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:27.154970 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:27.154987 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:27.158123 (+ 3136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":6847,"thread_start_us":6686,"threads_started":1}
I20260811 03:12:27.158689 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:27.141495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:54245)
0811 03:12:27.148720 (+ 7225us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:27.148753 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:27.148891 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:27.149672 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:27.149681 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:27.150225 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:27.150479 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:27.150494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.152354 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.152362 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:27.153289 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:27.153301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.154239 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.154244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:27.154455 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:27.155149 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:27.155167 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:27.158355 (+ 3188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":7075,"thread_start_us":6943,"threads_started":1}
W20260811 03:12:27.158743 21743 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:27.141604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:33601 (local address 127.11.60.62:39337)
0811 03:12:27.149135 (+ 7531us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:27.149142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:27.149157 (+ 15us) server_negotiation.cc:407] Connection header received
0811 03:12:27.149214 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:27.149217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:27.149292 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:27.149380 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:27.150575 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.152166 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.153411 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.153901 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.154003 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:27.158383 (+ 4380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:33601: BlockingRecv error: recv got EOF from 127.11.60.1:33601 (error 108)
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":7420,"spinlock_wait_cycles":6528,"thread_start_us":7850,"threads_started":1}
W20260811 03:12:27.159183 21742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:27.141612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:54245 (local address 127.11.60.62:39337)
0811 03:12:27.149269 (+ 7657us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:27.149273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:27.149288 (+ 15us) server_negotiation.cc:407] Connection header received
0811 03:12:27.149343 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:27.149347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:27.149480 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:27.149594 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:27.150642 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.152164 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.153464 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.154046 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.154201 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:27.158896 (+ 4695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:54245: BlockingRecv error: recv got EOF from 127.11.60.3:54245 (error 108)
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":7558,"spinlock_wait_cycles":22144,"thread_start_us":7360,"threads_started":1}
Aug 11 03:12:27 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:27 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 03:12:27.641790 21742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:27.631218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:39285 (local address 127.11.60.62:39337)
0811 03:12:27.631439 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:27.631448 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:27.631898 (+ 450us) server_negotiation.cc:407] Connection header received
0811 03:12:27.632097 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:27.632104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:27.632202 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:27.632319 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:27.633115 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.634501 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.635708 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.636560 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.636701 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:27.641601 (+ 4900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:39285: BlockingRecv error: recv got EOF from 127.11.60.2:39285 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 03:12:27.641881 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:27.631168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:39285)
0811 03:12:27.631765 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:27.631788 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:27.631958 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:27.632349 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:27.632352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:27.632753 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:27.632955 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:27.632963 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.634637 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.634641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:27.635555 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:27.635566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:27.636719 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:27.636723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:27.636839 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:27.637494 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:27.637512 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:27.641438 (+ 3926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":297,"threads_started":1}
Aug 11 03:12:28 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:28 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:28 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:28 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:28.171252 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:28.159858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:50033)
0811 03:12:28.161746 (+ 1888us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:28.161765 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:28.161914 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:28.162477 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:28.162481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:28.162909 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:28.163082 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:28.163091 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.164408 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.164414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:28.165666 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:28.165680 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.166730 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.166741 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:28.166932 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:28.167706 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:28.167737 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:28.170920 (+ 3183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1764,"thread_start_us":1616,"threads_started":1}
W20260811 03:12:28.171602 21748 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:28.159919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:50033 (local address 127.11.60.62:39337)
0811 03:12:28.161003 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:28.161011 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:28.161874 (+ 863us) server_negotiation.cc:407] Connection header received
0811 03:12:28.162078 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:28.162082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:28.162184 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:28.162328 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:28.163230 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.164241 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.165824 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.166554 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.166703 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:28.171239 (+ 4536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:50033: BlockingRecv error: recv got EOF from 127.11.60.1:50033 (error 108)
Metrics: {"server-negotiator.queue_time_us":972,"thread_start_us":845,"threads_started":1}
I20260811 03:12:28.172160 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:28.159913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:36587)
0811 03:12:28.160778 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:28.160808 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:28.160959 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:28.162873 (+ 1914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:28.162880 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:28.163429 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:28.163680 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:28.163695 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.165189 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.165193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:28.166534 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:28.166551 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.167636 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.167643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:28.167768 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:28.168351 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:28.168370 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:28.171899 (+ 3529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":546,"threads_started":1}
W20260811 03:12:28.172389 21751 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:28.160960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:36587 (local address 127.11.60.62:39337)
0811 03:12:28.162589 (+ 1629us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:28.162594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:28.162616 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:12:28.162650 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:28.162653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:28.162701 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:28.162833 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:28.163856 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.165024 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.166704 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.167398 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.167540 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:28.172130 (+ 4590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:36587: BlockingRecv error: recv got EOF from 127.11.60.3:36587 (error 108)
Metrics: {"server-negotiator.queue_time_us":1515,"thread_start_us":1350,"threads_started":1}
Aug 11 03:12:28 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:28 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:28.652019 21752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:28.642974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:60345)
0811 03:12:28.643485 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:28.643502 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:28.643625 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:28.643963 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:28.643967 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:28.644379 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:28.644542 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:28.644551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.646038 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.646047 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:28.647228 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:28.647242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.647934 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.647938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:28.648082 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:28.648716 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:28.648734 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:28.651665 (+ 2931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":290,"threads_started":1}
W20260811 03:12:28.652086 21751 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:28.643146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:60345 (local address 127.11.60.62:39337)
0811 03:12:28.643389 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:28.643396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:28.643585 (+ 189us) server_negotiation.cc:407] Connection header received
0811 03:12:28.643727 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:28.643730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:28.643793 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:28.643875 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:28.644682 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.645880 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.647349 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:28.647790 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:28.647886 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:28.651860 (+ 3974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:60345: BlockingRecv error: recv got EOF from 127.11.60.2:60345 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 03:12:29 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:29 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:29 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:29 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:29.182749 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:29.172282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:54827)
0811 03:12:29.172920 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:29.172943 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:29.173081 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:29.173677 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:29.173685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:29.174412 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:29.174679 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:29.174694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.176154 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.176160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:29.177095 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:29.177106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.178071 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.178076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:29.178210 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:29.178933 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:29.178951 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:29.182371 (+ 3420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":324,"threads_started":1}
W20260811 03:12:29.182893 21756 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:29.172501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:54827 (local address 127.11.60.62:39337)
0811 03:12:29.173258 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:29.173265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:29.173283 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:12:29.173344 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:29.173349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:29.173443 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:29.173577 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:29.174827 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.175981 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.177238 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.177908 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.178031 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:29.182616 (+ 4585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:54827: BlockingRecv error: recv got EOF from 127.11.60.1:54827 (error 108)
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":493,"threads_started":1}
I20260811 03:12:29.183223 21757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:29.173150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:55837)
0811 03:12:29.174529 (+ 1379us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:29.174559 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:29.174674 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:29.175382 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:29.175388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:29.175837 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:29.176038 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:29.176053 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.177046 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.177052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:29.178418 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:29.178431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.179131 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.179136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:29.179369 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:29.179967 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:29.179984 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:29.182923 (+ 2939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1232,"thread_start_us":1009,"threads_started":1}
W20260811 03:12:29.183614 21758 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:29.173284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:55837 (local address 127.11.60.62:39337)
0811 03:12:29.174887 (+ 1603us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:29.174897 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:29.174917 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:12:29.174972 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:29.174979 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:29.175084 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:29.175190 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:29.176161 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.176900 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.178560 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.178973 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.179100 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:29.183402 (+ 4302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:55837: BlockingRecv error: recv got EOF from 127.11.60.3:55837 (error 108)
Metrics: {"server-negotiator.queue_time_us":1390,"thread_start_us":1238,"threads_started":1}
Aug 11 03:12:29 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:29 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:29.661640 21759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:29.652910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:54283)
0811 03:12:29.653528 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:29.653544 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:29.653704 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:29.654044 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:29.654049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:29.654522 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:29.654699 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:29.654708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.655805 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.655812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:29.656740 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:29.656750 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.657492 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.657498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:29.657660 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:29.658352 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:29.658380 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:29.661297 (+ 2917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":309,"threads_started":1}
W20260811 03:12:29.661736 21758 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:29.653021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:54283 (local address 127.11.60.62:39337)
0811 03:12:29.653236 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:29.653244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:29.653641 (+ 397us) server_negotiation.cc:407] Connection header received
0811 03:12:29.653837 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:29.653841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:29.653911 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:29.654009 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:29.654851 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.655664 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.656892 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:29.657305 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:29.657435 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:29.661506 (+ 4071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:54283: BlockingRecv error: recv got EOF from 127.11.60.2:54283 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 03:12:30 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:30 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:30 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:30 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:30.195640 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:30.183804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:43001)
0811 03:12:30.184433 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:30.184491 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:30.184689 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:30.185601 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:30.185606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:30.186007 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:30.186199 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:30.186208 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.187823 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.187829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:30.189376 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:30.189392 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.190602 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.190611 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:30.190802 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:30.191695 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:30.191720 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:30.195252 (+ 3532us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":276,"threads_started":1}
I20260811 03:12:30.195793 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:30.184153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:42129)
0811 03:12:30.184671 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:30.184696 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:30.184817 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:30.185662 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:30.185668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:30.186093 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:30.186245 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:30.186254 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.187827 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.187832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:30.189349 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:30.189363 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.190617 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.190623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:30.190797 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:30.191614 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:30.191642 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:30.195417 (+ 3775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":241,"threads_started":1}
W20260811 03:12:30.195910 21763 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:30.184724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:42129 (local address 127.11.60.62:39337)
0811 03:12:30.185366 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:30.185370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:30.185384 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:12:30.185425 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:30.185428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:30.185487 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:30.185617 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:30.186448 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.187664 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.189512 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.190433 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.191177 (+ 744us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:30.195668 (+ 4491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:42129: BlockingRecv error: recv got EOF from 127.11.60.3:42129 (error 108)
Metrics: {"mutex_wait_us":727,"server-negotiator.queue_time_us":545,"thread_start_us":316,"threads_started":1}
W20260811 03:12:30.196261 21761 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:30.183899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:43001 (local address 127.11.60.62:39337)
0811 03:12:30.185280 (+ 1381us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:30.185285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:30.185299 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:12:30.185358 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:30.185362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:30.185424 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:30.185546 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:30.186400 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.187664 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.189512 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.190427 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.190579 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:30.195490 (+ 4911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:43001: BlockingRecv error: recv got EOF from 127.11.60.1:43001 (error 108)
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":1288,"spinlock_wait_cycles":70400,"thread_start_us":1108,"threads_started":1}
Aug 11 03:12:30 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:30 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:30.671365 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:30.662699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:41461)
0811 03:12:30.663260 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:30.663276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:30.663450 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:30.663795 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:30.663800 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:30.664221 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:30.664408 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:30.664417 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.665486 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.665493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:30.666450 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:30.666460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.667157 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.667163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:30.667335 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:30.668042 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:30.668060 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:30.671063 (+ 3003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":320,"threads_started":1}
W20260811 03:12:30.671695 21761 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:30.662803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:41461 (local address 127.11.60.62:39337)
0811 03:12:30.663101 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:30.663108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:30.663371 (+ 263us) server_negotiation.cc:407] Connection header received
0811 03:12:30.663581 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:30.663588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:30.663657 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:30.663760 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:30.664557 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.665328 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.666594 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:30.667003 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:30.667132 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:30.671317 (+ 4185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:41461: BlockingRecv error: recv got EOF from 127.11.60.2:41461 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 03:12:31 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:31 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:31 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.3@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:31 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.1@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:31.218338 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:31.196989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.3:54913)
0811 03:12:31.203831 (+ 6842us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:31.203863 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:31.204014 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:31.204961 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:31.204977 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:31.205489 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:31.205748 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:31.205763 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.207379 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.207384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:31.208860 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:31.208876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.209737 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.209742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:31.209864 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:31.214126 (+ 4262us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:31.214149 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:31.217933 (+ 3784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":6694,"thread_start_us":6580,"threads_started":1}
W20260811 03:12:31.218858 21767 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:31.197081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:54913 (local address 127.11.60.62:39337)
0811 03:12:31.204615 (+ 7534us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:31.204620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:31.204636 (+ 16us) server_negotiation.cc:407] Connection header received
0811 03:12:31.204681 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:31.204688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:31.204767 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:31.204911 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:31.205956 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.207241 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.209017 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.209592 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.209726 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:31.218540 (+ 8814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.3:54913: BlockingRecv error: recv got EOF from 127.11.60.3:54913 (error 108)
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":7413,"spinlock_wait_cycles":11776,"thread_start_us":6971,"threads_started":1}
W20260811 03:12:31.219453 21768 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:31.197077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:51179 (local address 127.11.60.62:39337)
0811 03:12:31.204424 (+ 7347us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:31.204428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:31.204449 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:12:31.204513 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:31.204517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:31.204578 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:31.204906 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:31.206378 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.207530 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.209245 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.209952 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.210037 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:31.219268 (+ 9231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.1:51179: BlockingRecv error: recv got EOF from 127.11.60.1:51179 (error 108)
Metrics: {"server-negotiator.queue_time_us":7249,"spinlock_wait_cycles":3968,"thread_start_us":7234,"threads_started":1}
I20260811 03:12:31.219660 21766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:31.196993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.1:51179)
0811 03:12:31.203542 (+ 6549us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:31.203568 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:31.203719 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:31.205473 (+ 1754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:31.205479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:31.206001 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:31.206214 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:31.206230 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.207663 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.207669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:31.209122 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:31.209134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.210120 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.210128 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:31.210272 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:31.214124 (+ 3852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:31.214149 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:31.218620 (+ 4471us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":6392,"thread_start_us":6270,"threads_started":1}
Aug 11 03:12:31 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 03:12:31 dist-test-slave-tm1g krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.60.2@KRBTEST.COM for oryx/127.11.60.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 03:12:31.681027 21770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:31.672559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39337 (local address 127.11.60.2:34437)
0811 03:12:31.673067 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:31.673083 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:31.673193 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:31.673484 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:31.673489 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:31.673883 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:31.674098 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:31.674106 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.675165 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.675169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:31.676106 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:31.676116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.676783 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.676788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:31.676902 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:31.677562 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:31.677579 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:31.680756 (+ 3177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.60.62:39337: Server oryx/127.11.60.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":260,"threads_started":1}
W20260811 03:12:31.681185 21768 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:31.672652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:34437 (local address 127.11.60.62:39337)
0811 03:12:31.672883 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:31.672889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:31.673163 (+ 274us) server_negotiation.cc:407] Connection header received
0811 03:12:31.673291 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:31.673294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:31.673358 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:31.673438 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:31.674239 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.675037 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.676236 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:31.676659 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:31.676756 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:31.680948 (+ 4192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.60.2:34437: BlockingRecv error: recv got EOF from 127.11.60.2:34437 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 03:12:32.001245 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 21255
I20260811 03:12:32.017216 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:32.002439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48534 (local address 127.11.60.1:41263)
0811 03:12:32.003096 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:32.003107 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:32.003138 (+ 31us) server_negotiation.cc:407] Connection header received
0811 03:12:32.003208 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:32.003214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:32.003443 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:32.003589 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:32.005001 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:32.006080 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:32.007327 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:32.007904 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:32.008008 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:32.014026 (+ 6018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:32.014036 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:32.014050 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:32.014089 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:32.016625 (+ 2536us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.60.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 03:12:32.016972 (+ 347us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48534: Request ticket server kudu/127.11.60.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":302,"threads_started":1}
W20260811 03:12:32.017341 21772 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48534: Request ticket server kudu/127.11.60.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 03:12:32.017422 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 21255: Not authorized: Client connection negotiation failed: client connection to 127.11.60.1:41263: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.60.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 03:12:32.017665 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 21255
I20260811 03:12:32.039073 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 21415
I20260811 03:12:32.052352 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:32.040100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51730 (local address 127.11.60.2:39803)
0811 03:12:32.040730 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:32.040737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:32.040769 (+ 32us) server_negotiation.cc:407] Connection header received
0811 03:12:32.040831 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:32.040838 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:32.041077 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:32.041234 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:32.042638 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:32.043745 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:32.045007 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:32.045680 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:32.045782 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:32.049101 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:32.049110 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:32.049125 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:32.049159 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:32.051833 (+ 2674us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.60.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 03:12:32.052117 (+ 284us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51730: Request ticket server kudu/127.11.60.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":282,"threads_started":1}
W20260811 03:12:32.052490 21773 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51730: Request ticket server kudu/127.11.60.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 03:12:32.052616 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 21415: Not authorized: Client connection negotiation failed: client connection to 127.11.60.2:39803: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.60.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 03:12:32.052814 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 21415
I20260811 03:12:32.072547 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 21556
I20260811 03:12:32.084753 21774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:32.073581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.11.60.3:44815)
0811 03:12:32.074225 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:32.074233 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:32.074261 (+ 28us) server_negotiation.cc:407] Connection header received
0811 03:12:32.074390 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:32.074397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:32.074644 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:32.074810 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:32.076087 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:32.077202 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:32.078518 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:32.079062 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:32.079166 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:32.081656 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:32.081665 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:32.081680 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:32.081724 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:32.084187 (+ 2463us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.60.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 03:12:32.084473 (+ 286us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37314: Request ticket server kudu/127.11.60.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":346,"threads_started":1}
W20260811 03:12:32.084864 21774 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37314: Request ticket server kudu/127.11.60.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 03:12:32.084967 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 21556: Not authorized: Client connection negotiation failed: client connection to 127.11.60.3:44815: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.60.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 03:12:32.085147 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 21556
I20260811 03:12:32.105057 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 21182
I20260811 03:12:32.106454 21247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:32.247505 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 21182
2026-08-11T03:12:32Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21650 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T03:12:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:12:32Z Disabled control of system clock
I20260811 03:12:32.339161 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42951
--webserver_interface=127.11.60.62
--webserver_port=0
--builtin_ntp_servers=127.11.60.20:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:42951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 03:12:32.684058 21780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:32.684350 21780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:32.684432 21780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:32.691694 21780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:12:32.691802 21780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:32.691840 21780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:32.691891 21780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:12:32.691934 21780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:12:32.701025 21780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:42951
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:32.702323 21780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:32.704120 21780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:32.716415 21785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:32.716501 21786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:32.716679 21788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:32.717305 21780 server_base.cc:1034] running on GCE node
I20260811 03:12:32.718147 21780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:32.719694 21780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:32.720908 21780 hybrid_clock.cc:648] HybridClock initialized: now 1786417952720856 us; error 46 us; skew 500 ppm
I20260811 03:12:32.721300 21780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:12:32.723769 21780 webserver.cc:460] Webserver started at http://127.11.60.62:44501/ using document root <none> and password file <none>
I20260811 03:12:32.724526 21780 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:32.724654 21780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:32.725025 21780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:32.728144 21780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "46ad65813ef84b078718ca2deed37ed3"
format_stamp: "Formatted at 2026-08-11 03:12:32 on dist-test-slave-tm1g"
server_key: "14ef2da0fcda78de9ee495a549a56f48"
server_key_iv: "24b8129f5faef4b09d8ec564c197c8f3"
server_key_version: "encryptionkey@0"
I20260811 03:12:32.729035 21780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "46ad65813ef84b078718ca2deed37ed3"
format_stamp: "Formatted at 2026-08-11 03:12:32 on dist-test-slave-tm1g"
server_key: "14ef2da0fcda78de9ee495a549a56f48"
server_key_iv: "24b8129f5faef4b09d8ec564c197c8f3"
server_key_version: "encryptionkey@0"
I20260811 03:12:32.734686 21780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 03:12:32.738498 21795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:32.740105 21780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:12:32.740300 21780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "46ad65813ef84b078718ca2deed37ed3"
format_stamp: "Formatted at 2026-08-11 03:12:32 on dist-test-slave-tm1g"
server_key: "14ef2da0fcda78de9ee495a549a56f48"
server_key_iv: "24b8129f5faef4b09d8ec564c197c8f3"
server_key_version: "encryptionkey@0"
I20260811 03:12:32.740556 21780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:32.759017 21780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:32.759963 21780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:32.760279 21780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:32.779309 21780 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:42951
I20260811 03:12:32.779333 21854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:42951 every 8 connection(s)
I20260811 03:12:32.780817 21780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:12:32.784750 21855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:32.785341 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 21780
I20260811 03:12:32.785558 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:12:32.786218 11504 external_mini_cluster.cc:1428] Setting key 3ec5078ad6f052f4b4cebf8f638f4562
I20260811 03:12:32.800038 21855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3: Bootstrap starting.
I20260811 03:12:32.802800 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:32.788474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39058 (local address 127.11.60.62:42951)
0811 03:12:32.789625 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:32.789640 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:32.789690 (+ 50us) server_negotiation.cc:407] Connection header received
0811 03:12:32.791779 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:32.791809 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:32.792444 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:32.792923 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:32.793569 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:32.795549 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:32.797084 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:32.798198 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:32.798401 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:32.799773 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:32.799786 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:32.799805 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:32.800549 (+ 744us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:32.800656 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:32.800683 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:32.800799 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:32.801034 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 03:12:32.801376 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":344,"threads_started":1}
I20260811 03:12:32.804023 21855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:32.805186 21855 log.cc:824] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:32.808221 21855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3: No bootstrap required, opened a new log
I20260811 03:12:32.816198 21855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad65813ef84b078718ca2deed37ed3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42951 } }
I20260811 03:12:32.816635 21855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:32.816749 21855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ad65813ef84b078718ca2deed37ed3, State: Initialized, Role: FOLLOWER
I20260811 03:12:32.817466 21855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad65813ef84b078718ca2deed37ed3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42951 } }
I20260811 03:12:32.817742 21855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:12:32.817902 21855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:12:32.818106 21855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:32.820463 21855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad65813ef84b078718ca2deed37ed3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42951 } }
I20260811 03:12:32.821066 21855 leader_election.cc:302] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46ad65813ef84b078718ca2deed37ed3; no voters:
I20260811 03:12:32.821771 21855 leader_election.cc:288] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:12:32.821897 21860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:32.822635 21860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [term 1 LEADER]: Becoming Leader. State: Replica: 46ad65813ef84b078718ca2deed37ed3, State: Running, Role: LEADER
I20260811 03:12:32.823275 21860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad65813ef84b078718ca2deed37ed3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42951 } }
I20260811 03:12:32.824319 21855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:12:32.826364 21862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ad65813ef84b078718ca2deed37ed3. Latest consensus state: current_term: 1 leader_uuid: "46ad65813ef84b078718ca2deed37ed3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad65813ef84b078718ca2deed37ed3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42951 } } }
I20260811 03:12:32.826870 21862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:32.826612 21861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46ad65813ef84b078718ca2deed37ed3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad65813ef84b078718ca2deed37ed3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42951 } } }
I20260811 03:12:32.827482 21861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:32.831465 21868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:12:32.837219 21868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:12:32.846467 21868 catalog_manager.cc:1347] Generated new cluster ID: 1a14e340caf9455ba80e5ef37aa9ef35
I20260811 03:12:32.846635 21868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:12:32.861443 21868 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:12:32.862812 21868 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:12:32.875231 21868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46ad65813ef84b078718ca2deed37ed3: Generated new TSK 0
I20260811 03:12:32.876102 21868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:12:32.889783 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42951
--builtin_ntp_servers=127.11.60.20:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 03:12:33.260450 21880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:33.260946 21880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:33.261140 21880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:33.277455 21880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:33.277617 21880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:33.277845 21880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:12:33.289793 21880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:33.291319 21880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:33.293171 21880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:33.307833 21887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:33.308137 21888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:33.309094 21880 server_base.cc:1034] running on GCE node
W20260811 03:12:33.309119 21890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:33.309878 21880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:33.311031 21880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:33.312319 21880 hybrid_clock.cc:648] HybridClock initialized: now 1786417953312246 us; error 62 us; skew 500 ppm
I20260811 03:12:33.312703 21880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:12:33.315421 21880 webserver.cc:460] Webserver started at http://127.11.60.1:44335/ using document root <none> and password file <none>
I20260811 03:12:33.316205 21880 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:33.316332 21880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:33.316695 21880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:33.319891 21880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "378e0ed503ab43a69195a484905b462c"
format_stamp: "Formatted at 2026-08-11 03:12:33 on dist-test-slave-tm1g"
server_key: "cf22430bb71b4467745d5ae2c711501a"
server_key_iv: "5b70793dd2314f823508b1ac7772357e"
server_key_version: "encryptionkey@0"
I20260811 03:12:33.320767 21880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "378e0ed503ab43a69195a484905b462c"
format_stamp: "Formatted at 2026-08-11 03:12:33 on dist-test-slave-tm1g"
server_key: "cf22430bb71b4467745d5ae2c711501a"
server_key_iv: "5b70793dd2314f823508b1ac7772357e"
server_key_version: "encryptionkey@0"
I20260811 03:12:33.326769 21880 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 03:12:33.330785 21896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:33.332319 21880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:12:33.332496 21880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "378e0ed503ab43a69195a484905b462c"
format_stamp: "Formatted at 2026-08-11 03:12:33 on dist-test-slave-tm1g"
server_key: "cf22430bb71b4467745d5ae2c711501a"
server_key_iv: "5b70793dd2314f823508b1ac7772357e"
server_key_version: "encryptionkey@0"
I20260811 03:12:33.332731 21880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:33.346797 21880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:33.347697 21880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:33.347993 21880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:33.349212 21880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:33.350805 21880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:33.350895 21880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:33.351060 21880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:33.351140 21880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:33.396927 21880 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:34839
I20260811 03:12:33.396972 22028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:34839 every 8 connection(s)
I20260811 03:12:33.398360 21880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:12:33.400198 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 21880
I20260811 03:12:33.400413 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:12:33.401104 11504 external_mini_cluster.cc:1428] Setting key e50869219d316e4d5e7770c8ed3b7a30
I20260811 03:12:33.404857 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42951
--builtin_ntp_servers=127.11.60.20:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 03:12:33.432476 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:33.403208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:49381 (local address 127.11.60.62:42951)
0811 03:12:33.418635 (+ 15427us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:33.418645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:33.425448 (+ 6803us) server_negotiation.cc:407] Connection header received
0811 03:12:33.425521 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:33.425525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:33.425603 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:33.425760 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:33.427052 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:33.427819 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:33.429427 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:33.429890 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:33.430034 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:33.431548 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:33.431553 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:33.431557 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:33.431577 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:33.431617 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:33.431629 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:33.431726 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:33.431965 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 03:12:33.432068 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15219,"thread_start_us":15437,"threads_started":1}
I20260811 03:12:33.433621 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:33.404098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42951 (local address 127.11.60.1:49381)
0811 03:12:33.423040 (+ 18942us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:33.423102 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:33.425050 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:33.426066 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:33.426077 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:33.426169 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:33.426866 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:33.426885 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:33.427992 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:33.427997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:33.429270 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:33.429282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:33.430139 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:33.430145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:33.430437 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:33.431194 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:12:33.431197 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:33.431240 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:12:33.431323 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:33.431728 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:33.431742 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:33.431751 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 03:12:33.431938 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 03:12:33.432411 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18266,"thread_start_us":17983,"threads_started":1}
I20260811 03:12:33.435257 22029 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42951
I20260811 03:12:33.435598 22029 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:33.436432 22029 heartbeater.cc:511] Master 127.11.60.62:42951 requested a full tablet report, sending...
I20260811 03:12:33.438692 21813 ts_manager.cc:194] Registered new tserver with Master: 378e0ed503ab43a69195a484905b462c (127.11.60.1:34839)
I20260811 03:12:33.440260 21813 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.1:49381
W20260811 03:12:33.806380 22034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:33.806679 22034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:33.806766 22034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:33.813980 22034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:33.814081 22034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:33.814186 22034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:12:33.823340 22034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:33.824666 22034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:33.826659 22034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:33.840204 22041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:33.840190 22039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:33.840345 22043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:33.842108 22034 server_base.cc:1034] running on GCE node
I20260811 03:12:33.842793 22034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:33.843716 22034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:33.844980 22034 hybrid_clock.cc:648] HybridClock initialized: now 1786417953844906 us; error 49 us; skew 500 ppm
I20260811 03:12:33.845353 22034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:12:33.847621 22034 webserver.cc:460] Webserver started at http://127.11.60.2:34279/ using document root <none> and password file <none>
I20260811 03:12:33.848348 22034 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:33.848486 22034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:33.848811 22034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:33.851747 22034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "30e0585e10e04d00be991f4893dedb52"
format_stamp: "Formatted at 2026-08-11 03:12:33 on dist-test-slave-tm1g"
server_key: "7a5d95de48ab5c557483fb623a54c69d"
server_key_iv: "b46a32bc6f8a53a4808364502a566aab"
server_key_version: "encryptionkey@0"
I20260811 03:12:33.852555 22034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "30e0585e10e04d00be991f4893dedb52"
format_stamp: "Formatted at 2026-08-11 03:12:33 on dist-test-slave-tm1g"
server_key: "7a5d95de48ab5c557483fb623a54c69d"
server_key_iv: "b46a32bc6f8a53a4808364502a566aab"
server_key_version: "encryptionkey@0"
I20260811 03:12:33.857767 22034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 03:12:33.861754 22051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:33.863180 22034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:12:33.863372 22034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "30e0585e10e04d00be991f4893dedb52"
format_stamp: "Formatted at 2026-08-11 03:12:33 on dist-test-slave-tm1g"
server_key: "7a5d95de48ab5c557483fb623a54c69d"
server_key_iv: "b46a32bc6f8a53a4808364502a566aab"
server_key_version: "encryptionkey@0"
I20260811 03:12:33.863600 22034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:33.884761 22034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:33.885643 22034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:33.885933 22034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:33.887051 22034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:33.888497 22034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:33.888626 22034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:33.888763 22034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:33.888855 22034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:33.922791 22034 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:41525
I20260811 03:12:33.923084 22175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:41525 every 8 connection(s)
I20260811 03:12:33.924348 22034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:12:33.933513 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 22034
I20260811 03:12:33.933677 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:12:33.934155 11504 external_mini_cluster.cc:1428] Setting key 5077bff46281767f5ea9d148107eecb7
I20260811 03:12:33.937603 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42951
--builtin_ntp_servers=127.11.60.20:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 03:12:33.938011 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:33.927105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:51577 (local address 127.11.60.62:42951)
0811 03:12:33.927323 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:33.927329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:33.928808 (+ 1479us) server_negotiation.cc:407] Connection header received
0811 03:12:33.930888 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:33.930893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:33.930966 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:33.931071 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:33.932194 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:33.933139 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:33.935043 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:33.935478 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:33.935574 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:33.937170 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:33.937174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:33.937178 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:33.937195 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:33.937228 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:33.937239 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:33.937311 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:33.937587 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 03:12:33.937663 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:12:33.939472 22178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:33.927681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42951 (local address 127.11.60.2:51577)
0811 03:12:33.928651 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:33.928688 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:33.930640 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:33.931323 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:33.931331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:33.931414 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:33.931999 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:33.932016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:33.933320 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:33.933323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:33.934874 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:33.934889 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:33.935698 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:33.935706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:33.935971 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:33.936810 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:12:33.936814 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:33.936867 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:12:33.936983 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:33.937347 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:33.937359 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:33.937370 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 03:12:33.937570 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 03:12:33.938030 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":280,"threads_started":1}
I20260811 03:12:33.940922 22176 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42951
I20260811 03:12:33.941238 22176 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:33.942004 22176 heartbeater.cc:511] Master 127.11.60.62:42951 requested a full tablet report, sending...
I20260811 03:12:33.943516 21813 ts_manager.cc:194] Registered new tserver with Master: 30e0585e10e04d00be991f4893dedb52 (127.11.60.2:41525)
I20260811 03:12:33.944800 21813 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.2:51577
W20260811 03:12:34.280190 22179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:34.280457 22179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:34.280614 22179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:34.287801 22179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:34.287901 22179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:34.288074 22179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:12:34.297061 22179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:34.298431 22179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:34.300263 22179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:34.313409 22184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:34.313446 22187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:34.313539 22185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:34.314265 22179 server_base.cc:1034] running on GCE node
I20260811 03:12:34.315021 22179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:34.315956 22179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:34.317162 22179 hybrid_clock.cc:648] HybridClock initialized: now 1786417954317120 us; error 30 us; skew 500 ppm
I20260811 03:12:34.317492 22179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:12:34.319689 22179 webserver.cc:460] Webserver started at http://127.11.60.3:36705/ using document root <none> and password file <none>
I20260811 03:12:34.320436 22179 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:34.320557 22179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:34.320892 22179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:34.323743 22179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "29abce5eb38343e8aef3466cba1d7002"
format_stamp: "Formatted at 2026-08-11 03:12:34 on dist-test-slave-tm1g"
server_key: "8783b6be417b2f56465366fb1fb1d0e1"
server_key_iv: "bdf9b7156fe00079303c74e72bae0e82"
server_key_version: "encryptionkey@0"
I20260811 03:12:34.324606 22179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "29abce5eb38343e8aef3466cba1d7002"
format_stamp: "Formatted at 2026-08-11 03:12:34 on dist-test-slave-tm1g"
server_key: "8783b6be417b2f56465366fb1fb1d0e1"
server_key_iv: "bdf9b7156fe00079303c74e72bae0e82"
server_key_version: "encryptionkey@0"
I20260811 03:12:34.329634 22179 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:12:34.333088 22193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:34.334295 22179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:12:34.334525 22179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "29abce5eb38343e8aef3466cba1d7002"
format_stamp: "Formatted at 2026-08-11 03:12:34 on dist-test-slave-tm1g"
server_key: "8783b6be417b2f56465366fb1fb1d0e1"
server_key_iv: "bdf9b7156fe00079303c74e72bae0e82"
server_key_version: "encryptionkey@0"
I20260811 03:12:34.334728 22179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:34.363077 22179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:34.363962 22179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:34.364262 22179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:34.365336 22179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:34.366865 22179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:34.367000 22179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:34.367161 22179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:34.367225 22179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:34.401276 22179 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:33219
I20260811 03:12:34.401319 22321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:33219 every 8 connection(s)
I20260811 03:12:34.402661 22179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:12:34.406700 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 22179
I20260811 03:12:34.406870 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:12:34.407321 11504 external_mini_cluster.cc:1428] Setting key ada99c946b51057c6c794cd1359bfacb
I20260811 03:12:34.415562 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:34.405283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:40111 (local address 127.11.60.62:42951)
0811 03:12:34.405521 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:34.405528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:34.407004 (+ 1476us) server_negotiation.cc:407] Connection header received
0811 03:12:34.409250 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:34.409257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:34.409342 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:34.409444 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:34.410781 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:34.411578 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:34.413057 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:34.413432 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:34.413520 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:34.414790 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:34.414796 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:34.414799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:34.414817 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:34.414852 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:34.414865 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:34.414946 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:34.415187 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 03:12:34.415269 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 03:12:34.416440 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:34.405798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42951 (local address 127.11.60.3:40111)
0811 03:12:34.406804 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:34.406862 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:34.408896 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:34.409716 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:34.409727 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:34.409841 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:34.410635 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:34.410654 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:34.411699 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:34.411703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:34.412917 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:34.412929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:34.413609 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:34.413613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:34.413794 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:34.414493 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:12:34.414496 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:34.414532 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:12:34.414605 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:34.414967 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:34.414974 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:34.414979 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 03:12:34.415112 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 03:12:34.415382 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":329,"threads_started":1}
I20260811 03:12:34.417784 22322 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42951
I20260811 03:12:34.418078 22322 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:34.418880 22322 heartbeater.cc:511] Master 127.11.60.62:42951 requested a full tablet report, sending...
I20260811 03:12:34.420311 21813 ts_manager.cc:194] Registered new tserver with Master: 29abce5eb38343e8aef3466cba1d7002 (127.11.60.3:33219)
I20260811 03:12:34.421211 21813 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.3:40111
I20260811 03:12:34.422894 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 03:12:34.429409 22031 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:34.427703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39070 (local address 127.11.60.62:42951)
0811 03:12:34.427887 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:34.427894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:34.428304 (+ 410us) server_negotiation.cc:407] Connection header received
0811 03:12:34.428498 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:34.428504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:34.428578 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:34.428680 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:34.429154 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39070: BlockingRecv error: recv got EOF from 127.0.0.1:39070 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 03:12:34.430438 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 21880
I20260811 03:12:34.441273 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:34.431348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57418 (local address 127.11.60.1:34839)
0811 03:12:34.434709 (+ 3361us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:34.434715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:34.434741 (+ 26us) server_negotiation.cc:407] Connection header received
0811 03:12:34.434796 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:34.434802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:34.434997 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:34.435170 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:34.435582 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:34.436582 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:34.437893 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:34.438587 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:34.438726 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:34.439843 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:34.439853 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:34.439865 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:34.440432 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:34.440490 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:34.440509 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:34.440645 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:34.440792 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 03:12:34.440927 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3219,"thread_start_us":3102,"threads_started":1}
I20260811 03:12:34.442591 22024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:34.526189 22029 heartbeater.cc:503] Master 127.11.60.62:42951 was elected leader, sending a full tablet report...
I20260811 03:12:34.527400 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 21880
W20260811 03:12:34.547310 21799 connection.cc:535] server connection from 127.11.60.1:49381 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 03:12:34.547578 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 22034
I20260811 03:12:34.555356 22336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:34.548496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57896 (local address 127.11.60.2:41525)
0811 03:12:34.549030 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:34.549045 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:34.549072 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:12:34.549132 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:34.549138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:34.549369 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:34.549496 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:34.549895 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:34.550976 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:34.552160 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:34.552714 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:34.552809 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:34.553828 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:34.553837 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:34.553849 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:34.554555 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:34.554602 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:34.554623 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:34.554711 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:34.554855 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 03:12:34.555005 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":276,"threads_started":1}
I20260811 03:12:34.556607 22171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:34.638242 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 22034
I20260811 03:12:34.658260 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 22179
I20260811 03:12:34.666457 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:34.659366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.11.60.3:33219)
0811 03:12:34.659925 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:34.659932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:34.659963 (+ 31us) server_negotiation.cc:407] Connection header received
0811 03:12:34.660024 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:34.660030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:34.660252 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:34.660378 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:34.660866 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:34.661920 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:34.663191 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:34.663764 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:34.663870 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:34.664866 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:34.664874 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:34.664888 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:34.665507 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:34.665585 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:34.665606 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:34.665710 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:34.665868 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 03:12:34.666001 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":261,"threads_started":1}
I20260811 03:12:34.667827 22316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:34.751631 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 22179
I20260811 03:12:34.770942 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 21780
I20260811 03:12:34.772249 21850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:34.877142 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 21780
2026-08-11T03:12:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2624 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T03:12:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:12:34Z Disabled control of system clock
I20260811 03:12:34.963784 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42771
--webserver_interface=127.11.60.62
--webserver_port=0
--builtin_ntp_servers=127.11.60.20:40515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:42771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 03:12:35.302294 22346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:35.302610 22346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:35.302763 22346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:35.309988 22346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:12:35.310108 22346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:35.310194 22346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:35.310309 22346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:12:35.310418 22346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:12:35.319567 22346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:42771
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:35.320912 22346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:35.322834 22346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:35.334787 22352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:35.334904 22351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:35.334966 22354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:35.336012 22346 server_base.cc:1034] running on GCE node
I20260811 03:12:35.336925 22346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:35.338457 22346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:35.339699 22346 hybrid_clock.cc:648] HybridClock initialized: now 1786417955339653 us; error 41 us; skew 500 ppm
I20260811 03:12:35.340066 22346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:12:35.342572 22346 webserver.cc:460] Webserver started at http://127.11.60.62:32891/ using document root <none> and password file <none>
I20260811 03:12:35.343312 22346 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:35.343456 22346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:35.343806 22346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:35.346879 22346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "b01cb1a5c66e4d589fe6e4beed2ff147"
format_stamp: "Formatted at 2026-08-11 03:12:35 on dist-test-slave-tm1g"
server_key: "ab55a5ffc4d3939bd7e1ebd24bff3297"
server_key_iv: "e8677f298a47894eb21933de9d768cfa"
server_key_version: "encryptionkey@0"
I20260811 03:12:35.347711 22346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "b01cb1a5c66e4d589fe6e4beed2ff147"
format_stamp: "Formatted at 2026-08-11 03:12:35 on dist-test-slave-tm1g"
server_key: "ab55a5ffc4d3939bd7e1ebd24bff3297"
server_key_iv: "e8677f298a47894eb21933de9d768cfa"
server_key_version: "encryptionkey@0"
I20260811 03:12:35.352999 22346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 03:12:35.356485 22362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:35.357788 22346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:12:35.357971 22346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "b01cb1a5c66e4d589fe6e4beed2ff147"
format_stamp: "Formatted at 2026-08-11 03:12:35 on dist-test-slave-tm1g"
server_key: "ab55a5ffc4d3939bd7e1ebd24bff3297"
server_key_iv: "e8677f298a47894eb21933de9d768cfa"
server_key_version: "encryptionkey@0"
I20260811 03:12:35.358209 22346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:35.364084 22346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:35.364876 22346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:35.365197 22346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:35.384318 22346 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:42771
I20260811 03:12:35.384352 22418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:42771 every 8 connection(s)
I20260811 03:12:35.385797 22346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:12:35.388757 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 22346
I20260811 03:12:35.388998 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:12:35.389846 11504 external_mini_cluster.cc:1428] Setting key 817f8fd5eef9b9b1fdcbc1f861d518bd
I20260811 03:12:35.390331 22419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:35.399039 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:35.392105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39000 (local address 127.11.60.62:42771)
0811 03:12:35.393136 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:35.393146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:35.393191 (+ 45us) server_negotiation.cc:407] Connection header received
0811 03:12:35.395229 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:35.395251 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:35.395691 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:35.396023 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:35.397249 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:35.397256 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:35.397268 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:35.397286 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:35.397355 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:35.397372 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:35.397572 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:35.397686 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 03:12:35.398013 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":395,"threads_started":1}
I20260811 03:12:35.404093 22419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147: Bootstrap starting.
I20260811 03:12:35.407198 22419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:35.408267 22419 log.cc:824] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:35.411257 22419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147: No bootstrap required, opened a new log
I20260811 03:12:35.418540 22419 raft_consensus.cc:348] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01cb1a5c66e4d589fe6e4beed2ff147" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42771 } }
I20260811 03:12:35.418931 22419 raft_consensus.cc:374] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:35.419028 22419 raft_consensus.cc:729] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b01cb1a5c66e4d589fe6e4beed2ff147, State: Initialized, Role: FOLLOWER
I20260811 03:12:35.419710 22419 consensus_queue.cc:260] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01cb1a5c66e4d589fe6e4beed2ff147" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42771 } }
I20260811 03:12:35.419960 22419 raft_consensus.cc:388] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:12:35.420087 22419 raft_consensus.cc:482] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:12:35.420266 22419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:35.422611 22419 raft_consensus.cc:504] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01cb1a5c66e4d589fe6e4beed2ff147" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42771 } }
I20260811 03:12:35.423113 22419 leader_election.cc:302] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b01cb1a5c66e4d589fe6e4beed2ff147; no voters:
I20260811 03:12:35.423719 22419 leader_election.cc:288] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:12:35.423877 22425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:35.424652 22425 raft_consensus.cc:686] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [term 1 LEADER]: Becoming Leader. State: Replica: b01cb1a5c66e4d589fe6e4beed2ff147, State: Running, Role: LEADER
I20260811 03:12:35.425256 22425 consensus_queue.cc:237] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01cb1a5c66e4d589fe6e4beed2ff147" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42771 } }
I20260811 03:12:35.425941 22419 sys_catalog.cc:564] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:12:35.428546 22426 sys_catalog.cc:455] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b01cb1a5c66e4d589fe6e4beed2ff147" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01cb1a5c66e4d589fe6e4beed2ff147" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42771 } } }
I20260811 03:12:35.429113 22426 sys_catalog.cc:458] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:35.429024 22427 sys_catalog.cc:455] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b01cb1a5c66e4d589fe6e4beed2ff147. Latest consensus state: current_term: 1 leader_uuid: "b01cb1a5c66e4d589fe6e4beed2ff147" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01cb1a5c66e4d589fe6e4beed2ff147" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42771 } } }
I20260811 03:12:35.429353 22427 sys_catalog.cc:458] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:35.434788 22432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:12:35.440152 22432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:12:35.449424 22432 catalog_manager.cc:1347] Generated new cluster ID: 7c563bce8b484bb2adfd04360c397af2
I20260811 03:12:35.449597 22432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:12:35.489123 22432 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:12:35.489647 22432 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:12:35.518765 22432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b01cb1a5c66e4d589fe6e4beed2ff147: Generated new TSK 0
I20260811 03:12:35.519936 22432 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:12:35.535120 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42771
--builtin_ntp_servers=127.11.60.20:40515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 03:12:35.889780 22444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:35.890108 22444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:35.890270 22444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:35.897521 22444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:35.897665 22444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:35.897861 22444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:12:35.907253 22444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:35.908910 22444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:35.910881 22444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:35.924628 22450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:35.924566 22451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:35.924537 22455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:35.925401 22444 server_base.cc:1034] running on GCE node
I20260811 03:12:35.926046 22444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:35.927078 22444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:35.928320 22444 hybrid_clock.cc:648] HybridClock initialized: now 1786417955928225 us; error 89 us; skew 500 ppm
I20260811 03:12:35.928637 22444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:12:35.931023 22444 webserver.cc:460] Webserver started at http://127.11.60.1:34261/ using document root <none> and password file <none>
I20260811 03:12:35.931741 22444 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:35.931878 22444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:35.932214 22444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:35.935313 22444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "0130f2ec7053456d95c9e199ab016b63"
format_stamp: "Formatted at 2026-08-11 03:12:35 on dist-test-slave-tm1g"
server_key: "7895dbf74e862b7e4aedcfbfd0ad3cad"
server_key_iv: "8a7933775208cfc058a4c256cb016cc3"
server_key_version: "encryptionkey@0"
I20260811 03:12:35.936111 22444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "0130f2ec7053456d95c9e199ab016b63"
format_stamp: "Formatted at 2026-08-11 03:12:35 on dist-test-slave-tm1g"
server_key: "7895dbf74e862b7e4aedcfbfd0ad3cad"
server_key_iv: "8a7933775208cfc058a4c256cb016cc3"
server_key_version: "encryptionkey@0"
I20260811 03:12:35.941167 22444 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:12:35.944603 22461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:35.945888 22444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 03:12:35.946080 22444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "0130f2ec7053456d95c9e199ab016b63"
format_stamp: "Formatted at 2026-08-11 03:12:35 on dist-test-slave-tm1g"
server_key: "7895dbf74e862b7e4aedcfbfd0ad3cad"
server_key_iv: "8a7933775208cfc058a4c256cb016cc3"
server_key_version: "encryptionkey@0"
I20260811 03:12:35.946303 22444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:35.951866 22444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:35.952626 22444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:35.952900 22444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:35.953876 22444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:35.955325 22444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:35.955452 22444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:35.955590 22444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:35.955680 22444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:35.988952 22444 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:44209
I20260811 03:12:35.989012 22581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:44209 every 8 connection(s)
I20260811 03:12:35.990338 22444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:12:35.994356 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 22444
I20260811 03:12:35.994587 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:12:35.995236 11504 external_mini_cluster.cc:1428] Setting key 52bff1dd64ac015460c7e595fa871687
I20260811 03:12:35.998104 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42771
--builtin_ntp_servers=127.11.60.20:40515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 03:12:35.999432 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:35.993316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42771 (local address 127.11.60.1:38079)
0811 03:12:35.994184 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:35.994218 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:35.996033 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:35.996748 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:35.996757 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:35.996841 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:35.997615 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:12:35.997617 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:35.997654 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:12:35.997735 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:35.998096 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:35.998109 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:35.998117 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 03:12:35.998415 (+ 298us) client_negotiation.cc:240] Negotiation successful
0811 03:12:35.998607 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":268,"threads_started":1}
I20260811 03:12:36.000186 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:35.992903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:38079 (local address 127.11.60.62:42771)
0811 03:12:35.993374 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:35.993378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:35.994361 (+ 983us) server_negotiation.cc:407] Connection header received
0811 03:12:35.996290 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:35.996295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:35.996370 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:35.996498 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:35.997898 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:35.997905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:35.997914 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:35.997929 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:35.997967 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:35.997979 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:35.998130 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:35.999863 (+ 1733us) server_negotiation.cc:299] Negotiation successful
0811 03:12:35.999938 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":259,"threads_started":1}
I20260811 03:12:36.001350 22582 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42771
I20260811 03:12:36.001654 22582 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:36.002218 22582 heartbeater.cc:511] Master 127.11.60.62:42771 requested a full tablet report, sending...
I20260811 03:12:36.004412 22382 ts_manager.cc:194] Registered new tserver with Master: 0130f2ec7053456d95c9e199ab016b63 (127.11.60.1:44209)
W20260811 03:12:36.331321 22586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:36.331578 22586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:36.331667 22586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:36.338874 22586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:36.338977 22586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:36.339094 22586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:12:36.348001 22586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:36.349275 22586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:36.351137 22586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:36.364377 22595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:36.364327 22593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:36.364427 22592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:36.367794 22586 server_base.cc:1034] running on GCE node
I20260811 03:12:36.368408 22586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:36.369457 22586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:36.370674 22586 hybrid_clock.cc:648] HybridClock initialized: now 1786417956370585 us; error 79 us; skew 500 ppm
I20260811 03:12:36.370993 22586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:12:36.376317 22586 webserver.cc:460] Webserver started at http://127.11.60.2:44841/ using document root <none> and password file <none>
I20260811 03:12:36.377043 22586 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:36.377175 22586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:36.377508 22586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:36.380439 22586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "fe83a260b3a8432dbcb757348c850266"
format_stamp: "Formatted at 2026-08-11 03:12:36 on dist-test-slave-tm1g"
server_key: "181be291d69f150951a130214bb30d43"
server_key_iv: "1f1014ee720fb2c155cbea9408630ce6"
server_key_version: "encryptionkey@0"
I20260811 03:12:36.381237 22586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "fe83a260b3a8432dbcb757348c850266"
format_stamp: "Formatted at 2026-08-11 03:12:36 on dist-test-slave-tm1g"
server_key: "181be291d69f150951a130214bb30d43"
server_key_iv: "1f1014ee720fb2c155cbea9408630ce6"
server_key_version: "encryptionkey@0"
I20260811 03:12:36.386477 22586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 03:12:36.389829 22601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:36.391249 22586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:12:36.391403 22586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "fe83a260b3a8432dbcb757348c850266"
format_stamp: "Formatted at 2026-08-11 03:12:36 on dist-test-slave-tm1g"
server_key: "181be291d69f150951a130214bb30d43"
server_key_iv: "1f1014ee720fb2c155cbea9408630ce6"
server_key_version: "encryptionkey@0"
I20260811 03:12:36.391629 22586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:36.397236 22586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:36.397984 22586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:36.398247 22586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:36.399219 22586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:36.400581 22586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:36.400701 22586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:36.400825 22586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:36.400915 22586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:36.434006 22586 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:43351
I20260811 03:12:36.434113 22721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:43351 every 8 connection(s)
I20260811 03:12:36.435424 22586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:12:36.436424 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 22586
I20260811 03:12:36.436635 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:12:36.437186 11504 external_mini_cluster.cc:1428] Setting key 3231c8bbfcb53f237b8b1a0b61992769
I20260811 03:12:36.440501 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42771
--builtin_ntp_servers=127.11.60.20:40515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 03:12:36.445369 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:36.438439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:57047 (local address 127.11.60.62:42771)
0811 03:12:36.438660 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:36.438667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:36.440213 (+ 1546us) server_negotiation.cc:407] Connection header received
0811 03:12:36.442948 (+ 2735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:36.442953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:36.443014 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:36.443096 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:36.444600 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:36.444605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:36.444616 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:36.444633 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:36.444664 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:36.444676 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:36.444798 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:36.445000 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 03:12:36.445072 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:12:36.446245 22725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:36.438903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42771 (local address 127.11.60.2:57047)
0811 03:12:36.440020 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:36.440092 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:36.442656 (+ 2564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:36.443349 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:36.443362 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:36.443478 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:36.444299 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:12:36.444302 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:36.444339 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:12:36.444423 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:36.444773 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:36.444784 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:36.444792 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 03:12:36.445074 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 03:12:36.445330 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":365,"threads_started":1}
I20260811 03:12:36.447655 22722 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42771
I20260811 03:12:36.447930 22722 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:36.448446 22722 heartbeater.cc:511] Master 127.11.60.62:42771 requested a full tablet report, sending...
I20260811 03:12:36.449981 22382 ts_manager.cc:194] Registered new tserver with Master: fe83a260b3a8432dbcb757348c850266 (127.11.60.2:43351)
W20260811 03:12:36.778774 22726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:36.779115 22726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:36.779291 22726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:36.787070 22726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:36.787200 22726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:36.787388 22726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:12:36.796806 22726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:42771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:36.798197 22726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:36.800014 22726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:36.814481 22733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:36.814601 22734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:36.814800 22736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:36.815671 22726 server_base.cc:1034] running on GCE node
I20260811 03:12:36.816293 22726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:36.817267 22726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:36.818491 22726 hybrid_clock.cc:648] HybridClock initialized: now 1786417956818455 us; error 56 us; skew 500 ppm
I20260811 03:12:36.818823 22726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:12:36.821141 22726 webserver.cc:460] Webserver started at http://127.11.60.3:32809/ using document root <none> and password file <none>
I20260811 03:12:36.821897 22726 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:36.822046 22726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:36.822443 22726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:36.825407 22726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "44cb7922fce14b44b1b0d56e7c0aedaa"
format_stamp: "Formatted at 2026-08-11 03:12:36 on dist-test-slave-tm1g"
server_key: "075fe8d072152b5bd1a18635310bea97"
server_key_iv: "b325e0c28de8dd62c4d39129b325c684"
server_key_version: "encryptionkey@0"
I20260811 03:12:36.826406 22726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "44cb7922fce14b44b1b0d56e7c0aedaa"
format_stamp: "Formatted at 2026-08-11 03:12:36 on dist-test-slave-tm1g"
server_key: "075fe8d072152b5bd1a18635310bea97"
server_key_iv: "b325e0c28de8dd62c4d39129b325c684"
server_key_version: "encryptionkey@0"
I20260811 03:12:36.832116 22726 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:12:36.835896 22743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:36.837229 22726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:12:36.837414 22726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "44cb7922fce14b44b1b0d56e7c0aedaa"
format_stamp: "Formatted at 2026-08-11 03:12:36 on dist-test-slave-tm1g"
server_key: "075fe8d072152b5bd1a18635310bea97"
server_key_iv: "b325e0c28de8dd62c4d39129b325c684"
server_key_version: "encryptionkey@0"
I20260811 03:12:36.837651 22726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:36.843397 22726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:36.844213 22726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:36.844496 22726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:36.845474 22726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:36.846993 22726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:36.847127 22726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:36.847265 22726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:36.847353 22726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:36.880623 22726 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:37157
I20260811 03:12:36.880713 22863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:37157 every 8 connection(s)
I20260811 03:12:36.881994 22726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:12:36.889067 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 22726
I20260811 03:12:36.889287 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:12:36.889863 11504 external_mini_cluster.cc:1428] Setting key 2d75c2fa583f0171fb8bac1f1b21c0bd
I20260811 03:12:36.890605 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:36.884811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:52955 (local address 127.11.60.62:42771)
0811 03:12:36.885033 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:36.885040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:36.886476 (+ 1436us) server_negotiation.cc:407] Connection header received
0811 03:12:36.888139 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:36.888145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:36.888214 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:36.888314 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:36.889786 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:36.889790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:36.889793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:36.889810 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:36.889844 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:36.889855 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:36.889981 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:36.890212 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 03:12:36.890267 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:12:36.891719 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:36.885421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:42771 (local address 127.11.60.3:52955)
0811 03:12:36.886318 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:36.886360 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:36.887879 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:36.888514 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:36.888523 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:36.888607 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:36.889422 (+ 815us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:12:36.889425 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:36.889480 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:12:36.889601 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:36.889966 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:36.889978 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:36.889987 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 03:12:36.890292 (+ 305us) client_negotiation.cc:240] Negotiation successful
0811 03:12:36.890579 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":291,"threads_started":1}
I20260811 03:12:36.893205 22864 heartbeater.cc:347] Connected to a master server at 127.11.60.62:42771
I20260811 03:12:36.893467 22864 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:36.893990 22864 heartbeater.cc:511] Master 127.11.60.62:42771 requested a full tablet report, sending...
I20260811 03:12:36.895378 22382 ts_manager.cc:194] Registered new tserver with Master: 44cb7922fce14b44b1b0d56e7c0aedaa (127.11.60.3:37157)
I20260811 03:12:36.905500 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 03:12:36.911592 22584 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:12:36.909985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.11.60.62:42771)
0811 03:12:36.910209 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:36.910218 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:36.910706 (+ 488us) server_negotiation.cc:407] Connection header received
0811 03:12:36.910829 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:36.910833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:36.910885 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:36.910969 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:36.911343 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39004: BlockingRecv error: recv got EOF from 127.0.0.1:39004 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 03:12:36.912086 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 22444
I20260811 03:12:36.915721 22873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:36.913152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49034 (local address 127.11.60.1:44209)
0811 03:12:36.913623 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:36.913630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:36.913654 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:12:36.913707 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:36.913712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:36.913904 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:36.914026 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:36.914963 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:36.914969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:36.914980 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:36.914999 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:36.915042 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:36.915057 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:36.915164 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:36.915316 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 03:12:36.915422 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":222,"threads_started":1}
I20260811 03:12:36.916993 22577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:36.999143 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 22444
I20260811 03:12:37.018958 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 22586
I20260811 03:12:37.022692 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:37.019915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43904 (local address 127.11.60.2:43351)
0811 03:12:37.020466 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:37.020474 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:37.020501 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:12:37.020563 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:37.020570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:37.020790 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:37.020917 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:37.021867 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:37.021899 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:37.021921 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:37.021954 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:37.021999 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:37.022017 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:37.022134 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:37.022216 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 03:12:37.022357 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":267,"threads_started":1}
I20260811 03:12:37.023967 22716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:37.104720 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 22586
I20260811 03:12:37.122751 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 22726
I20260811 03:12:37.126847 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:37.123755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.11.60.3:37157)
0811 03:12:37.124373 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:37.124380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:37.124412 (+ 32us) server_negotiation.cc:407] Connection header received
0811 03:12:37.124493 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:37.124499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:37.124731 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:37.124857 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:37.126003 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:37.126010 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:37.126021 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:12:37.126038 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:37.126079 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:12:37.126093 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:37.126226 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:37.126279 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 03:12:37.126441 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":310,"threads_started":1}
I20260811 03:12:37.128198 22858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:37.213011 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 22726
I20260811 03:12:37.233872 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 22346
I20260811 03:12:37.235280 22414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:37.339298 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 22346
2026-08-11T03:12:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2459 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:12:37 dist-test-slave-tm1g krb5kdc[22884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:12:37 dist-test-slave-tm1g krb5kdc[22884](info): set up 2 sockets
Aug 11 03:12:37 dist-test-slave-tm1g krb5kdc[22884](info): commencing operation
krb5kdc: starting...
W20260811 03:12:38.512100 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:12:38 dist-test-slave-tm1g krb5kdc[22884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:12:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:12:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:12:38.815358 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:36745
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:36745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:39.160962 22897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:39.161253 22897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:39.161345 22897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:39.168708 22897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:12:39.168824 22897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:39.168872 22897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:39.168924 22897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:12:39.168969 22897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:12:39.179123 22897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:36745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:36745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:39.180521 22897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:39.181851 22897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:39.194574 22903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:39.194546 22904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:39.194628 22906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:39.196249 22897 server_base.cc:1034] running on GCE node
I20260811 03:12:39.197021 22897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:39.198541 22897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:39.199788 22897 hybrid_clock.cc:648] HybridClock initialized: now 1786417959199742 us; error 55 us; skew 500 ppm
I20260811 03:12:39.200155 22897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:39 dist-test-slave-tm1g krb5kdc[22884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417959, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:39.203410 22897 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:12:39.204960 22897 webserver.cc:460] Webserver started at http://127.11.60.62:40821/ using document root <none> and password file <none>
I20260811 03:12:39.205698 22897 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:39.205852 22897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:39.206182 22897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:39.209268 22897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "8b3c8d22f1034908a72132eec376b109"
format_stamp: "Formatted at 2026-08-11 03:12:39 on dist-test-slave-tm1g"
server_key: "c5770cc51fb088c281a99c738d2c8a38"
server_key_iv: "5d509802a10c5585e8be989819ed61eb"
server_key_version: "encryptionkey@0"
I20260811 03:12:39.210038 22897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "8b3c8d22f1034908a72132eec376b109"
format_stamp: "Formatted at 2026-08-11 03:12:39 on dist-test-slave-tm1g"
server_key: "c5770cc51fb088c281a99c738d2c8a38"
server_key_iv: "5d509802a10c5585e8be989819ed61eb"
server_key_version: "encryptionkey@0"
I20260811 03:12:39.215471 22897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 03:12:39.218957 22914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:39.220270 22897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:12:39.220460 22897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "8b3c8d22f1034908a72132eec376b109"
format_stamp: "Formatted at 2026-08-11 03:12:39 on dist-test-slave-tm1g"
server_key: "c5770cc51fb088c281a99c738d2c8a38"
server_key_iv: "5d509802a10c5585e8be989819ed61eb"
server_key_version: "encryptionkey@0"
I20260811 03:12:39.220669 22897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:39.233464 22897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:39.237612 22897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:39.237915 22897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:39.256745 22897 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:36745
I20260811 03:12:39.256776 22969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:36745 every 8 connection(s)
I20260811 03:12:39.258253 22897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:12:39.262063 22970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:39.265367 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 22897
I20260811 03:12:39.265630 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:12:39.266453 11504 external_mini_cluster.cc:1428] Setting key ef5d26ef359aa2e8ab83b659a706a012
I20260811 03:12:39.275432 22970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109: Bootstrap starting.
I20260811 03:12:39.279290 22970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:39.280691 22970 log.cc:824] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109: Log is configured to *not* fsync() on all Append() calls
Aug 11 03:12:39 dist-test-slave-tm1g krb5kdc[22884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:12:39.284243 22970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109: No bootstrap required, opened a new log
I20260811 03:12:39.291014 22973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:39.268635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.11.60.62:36745)
0811 03:12:39.269438 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:39.269449 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:39.269490 (+ 41us) server_negotiation.cc:407] Connection header received
0811 03:12:39.271211 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:39.271234 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:39.271658 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:39.271993 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:39.273922 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:39.275267 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:39.277091 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:39.277807 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:39.277960 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:39.283271 (+ 5311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:39.283278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:39.283293 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:39.283329 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:39.286727 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:39.287543 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:39.287550 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:39.287557 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:39.287735 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:39.288167 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:39.288170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:39.288172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:39.288713 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:39.288995 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:39.289429 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 03:12:39.289832 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":381,"threads_started":1}
I20260811 03:12:39.293664 22970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b3c8d22f1034908a72132eec376b109" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36745 } }
I20260811 03:12:39.294124 22970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:39.294265 22970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b3c8d22f1034908a72132eec376b109, State: Initialized, Role: FOLLOWER
I20260811 03:12:39.294881 22970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b3c8d22f1034908a72132eec376b109" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36745 } }
I20260811 03:12:39.295096 22970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:12:39.295253 22970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:12:39.295421 22970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:39.297879 22970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b3c8d22f1034908a72132eec376b109" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36745 } }
I20260811 03:12:39.298436 22970 leader_election.cc:302] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b3c8d22f1034908a72132eec376b109; no voters:
I20260811 03:12:39.299095 22970 leader_election.cc:288] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:12:39.299273 22975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:39.300395 22975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [term 1 LEADER]: Becoming Leader. State: Replica: 8b3c8d22f1034908a72132eec376b109, State: Running, Role: LEADER
I20260811 03:12:39.300941 22975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b3c8d22f1034908a72132eec376b109" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36745 } }
I20260811 03:12:39.301374 22970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:12:39.303818 22977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b3c8d22f1034908a72132eec376b109. Latest consensus state: current_term: 1 leader_uuid: "8b3c8d22f1034908a72132eec376b109" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b3c8d22f1034908a72132eec376b109" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36745 } } }
I20260811 03:12:39.304318 22977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:39.303933 22976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b3c8d22f1034908a72132eec376b109" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b3c8d22f1034908a72132eec376b109" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36745 } } }
I20260811 03:12:39.304694 22976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109 [sys.catalog]: This master's current role is: LEADER
I20260811 03:12:39.307334 22983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:12:39.312865 22983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:12:39.322084 22983 catalog_manager.cc:1347] Generated new cluster ID: 5fd750a274e94f408fa0d5b25295f1ee
I20260811 03:12:39.322206 22983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:12:39.336871 22983 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:12:39.338271 22983 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:12:39.347568 22983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109: Generated new TSK 0
I20260811 03:12:39.348760 22983 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:12:39.419763 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36745
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:39.770671 22996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:39.770946 22996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:39.771040 22996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:39.778334 22996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:39.778502 22996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:39.778625 22996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:12:39.788381 22996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:39.789700 22996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:39.791101 22996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:39.804325 23002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:39.804296 23004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:39.804602 23001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:39.805227 22996 server_base.cc:1034] running on GCE node
I20260811 03:12:39.805938 22996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:39.807053 22996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:39.808285 22996 hybrid_clock.cc:648] HybridClock initialized: now 1786417959808262 us; error 48 us; skew 500 ppm
I20260811 03:12:39.808606 22996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:39 dist-test-slave-tm1g krb5kdc[22884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417959, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:39.811820 22996 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:12:39.813215 22996 webserver.cc:460] Webserver started at http://127.11.60.1:37553/ using document root <none> and password file <none>
I20260811 03:12:39.813951 22996 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:39.814100 22996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:39.814474 22996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:39.817386 22996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "039942daf13041d3b4f09c7e9d2dd436"
format_stamp: "Formatted at 2026-08-11 03:12:39 on dist-test-slave-tm1g"
server_key: "1e095fd2965fef766bdc13ad621e67c7"
server_key_iv: "69d6d28e227b085bb4ea8b527b858aec"
server_key_version: "encryptionkey@0"
I20260811 03:12:39.818239 22996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "039942daf13041d3b4f09c7e9d2dd436"
format_stamp: "Formatted at 2026-08-11 03:12:39 on dist-test-slave-tm1g"
server_key: "1e095fd2965fef766bdc13ad621e67c7"
server_key_iv: "69d6d28e227b085bb4ea8b527b858aec"
server_key_version: "encryptionkey@0"
I20260811 03:12:39.823421 22996 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:12:39.827015 23013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:39.828361 22996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:12:39.828548 22996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "039942daf13041d3b4f09c7e9d2dd436"
format_stamp: "Formatted at 2026-08-11 03:12:39 on dist-test-slave-tm1g"
server_key: "1e095fd2965fef766bdc13ad621e67c7"
server_key_iv: "69d6d28e227b085bb4ea8b527b858aec"
server_key_version: "encryptionkey@0"
I20260811 03:12:39.828747 22996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:39.846839 22996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:39.851347 22996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:39.851655 22996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:39.852586 22996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:39.853929 22996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:39.854033 22996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:39.854182 22996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:39.854254 22996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:39.888841 22996 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:36847
I20260811 03:12:39.888888 23133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:36847 every 8 connection(s)
I20260811 03:12:39.890143 22996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:12:39.899722 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 22996
I20260811 03:12:39.900003 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:12:39.900664 11504 external_mini_cluster.cc:1428] Setting key 342375f8bc75c55c41f6398748344ded
Aug 11 03:12:39 dist-test-slave-tm1g krb5kdc[22884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417959, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:12:39.910367 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:39.892750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:59345 (local address 127.11.60.62:36745)
0811 03:12:39.893243 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:39.893247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:39.893939 (+ 692us) server_negotiation.cc:407] Connection header received
0811 03:12:39.895892 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:39.895904 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:39.896013 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:39.896139 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:39.897646 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:39.898481 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:39.900168 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:39.900601 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:39.900748 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:39.904590 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:39.904596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:39.904599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:39.904629 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:39.907303 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:39.908024 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:39.908037 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:39.908041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:39.908170 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:39.908601 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:39.908607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:39.908610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:39.908876 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:39.909036 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:39.909715 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 03:12:39.909942 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":282,"threads_started":1}
I20260811 03:12:39.911357 23138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:39.893090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:36745 (local address 127.11.60.1:59345)
0811 03:12:39.893779 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:39.893823 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:39.895584 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:39.896374 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:39.896383 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:39.896885 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:39.897445 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:39.897462 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:39.898661 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:39.898665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:39.900006 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:39.900019 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:39.900851 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:39.900860 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:39.901107 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:39.901910 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:39.901938 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:39.904365 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:39.907434 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:39.907443 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:39.907459 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:39.907880 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:39.908291 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:39.908296 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:39.908300 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:39.908488 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:39.909014 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:39.909022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:39.909353 (+ 331us) client_negotiation.cc:769] Sending connection context
0811 03:12:39.909622 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 03:12:39.910165 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":275,"threads_started":1}
I20260811 03:12:39.912943 23134 heartbeater.cc:347] Connected to a master server at 127.11.60.62:36745
I20260811 03:12:39.913239 23134 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:39.913946 23134 heartbeater.cc:511] Master 127.11.60.62:36745 requested a full tablet report, sending...
I20260811 03:12:39.915994 22933 ts_manager.cc:194] Registered new tserver with Master: 039942daf13041d3b4f09c7e9d2dd436 (127.11.60.1:36847)
I20260811 03:12:39.917415 22933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:59345
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:12:39.962131 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36745
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:40.300894 23141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:40.301175 23141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:40.301321 23141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:40.308959 23141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:40.309062 23141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:40.309175 23141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:12:40.318763 23141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:40.319994 23141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:40.321252 23141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:40.335016 23146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:40.335089 23147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:40.335059 23149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:40.335937 23141 server_base.cc:1034] running on GCE node
I20260811 03:12:40.336608 23141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:40.337497 23141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:40.338711 23141 hybrid_clock.cc:648] HybridClock initialized: now 1786417960338673 us; error 29 us; skew 500 ppm
I20260811 03:12:40.339033 23141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:40 dist-test-slave-tm1g krb5kdc[22884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417960, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:40.342185 23141 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:12:40.343714 23141 webserver.cc:460] Webserver started at http://127.11.60.2:46821/ using document root <none> and password file <none>
I20260811 03:12:40.344457 23141 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:40.344600 23141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:40.344925 23141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:40.347987 23141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "a127b3e446a14a3089ec8d22a3876b23"
format_stamp: "Formatted at 2026-08-11 03:12:40 on dist-test-slave-tm1g"
server_key: "f97c8a8523bfb3d69aa00632420bf74d"
server_key_iv: "bd8dd1ff7a020a08f7d2fdcc98218add"
server_key_version: "encryptionkey@0"
I20260811 03:12:40.348774 23141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "a127b3e446a14a3089ec8d22a3876b23"
format_stamp: "Formatted at 2026-08-11 03:12:40 on dist-test-slave-tm1g"
server_key: "f97c8a8523bfb3d69aa00632420bf74d"
server_key_iv: "bd8dd1ff7a020a08f7d2fdcc98218add"
server_key_version: "encryptionkey@0"
I20260811 03:12:40.353847 23141 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:12:40.357349 23158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:40.358767 23141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:12:40.358899 23141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "a127b3e446a14a3089ec8d22a3876b23"
format_stamp: "Formatted at 2026-08-11 03:12:40 on dist-test-slave-tm1g"
server_key: "f97c8a8523bfb3d69aa00632420bf74d"
server_key_iv: "bd8dd1ff7a020a08f7d2fdcc98218add"
server_key_version: "encryptionkey@0"
I20260811 03:12:40.359066 23141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:40.387985 23141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:40.392544 23141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:40.392805 23141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:40.393724 23141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:40.395179 23141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:40.395253 23141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:40.395340 23141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:40.395394 23141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:40.430135 23141 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:46165
I20260811 03:12:40.430229 23279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:46165 every 8 connection(s)
I20260811 03:12:40.431519 23141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:12:40.438490 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 23141
I20260811 03:12:40.438695 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:12:40.439144 11504 external_mini_cluster.cc:1428] Setting key d356a0af099599fcb08a2c186821dd67
Aug 11 03:12:40 dist-test-slave-tm1g krb5kdc[22884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417960, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:12:40.453812 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:40.434352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:53399 (local address 127.11.60.62:36745)
0811 03:12:40.434784 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:40.434788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:40.435759 (+ 971us) server_negotiation.cc:407] Connection header received
0811 03:12:40.437775 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:40.437781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:40.437876 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:40.438005 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:40.439817 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:40.440813 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:40.442449 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:40.443237 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:40.443388 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:40.447008 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:40.447017 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:40.447024 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:40.447070 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:40.450702 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:40.451399 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:40.451407 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:40.451413 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:40.451584 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:40.451989 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:40.451996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:40.452003 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:40.452374 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:40.452540 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:40.453193 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 03:12:40.453432 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":215,"threads_started":1}
I20260811 03:12:40.454985 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:40.434897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:36745 (local address 127.11.60.2:53399)
0811 03:12:40.435599 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:40.435643 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:40.437496 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:40.438247 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:40.438255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:40.438953 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:40.439643 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:40.439661 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:40.440959 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:40.440971 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:40.442248 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:40.442259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:40.443442 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:40.443447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:40.443662 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:40.444338 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:40.444359 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:40.446774 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:40.450846 (+ 4072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:40.450852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:40.450864 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:40.451264 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:40.451693 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:40.451696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:40.451699 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:40.451870 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:40.452508 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:40.452513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:40.452815 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 03:12:40.453074 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 03:12:40.453608 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":284,"threads_started":1}
I20260811 03:12:40.456415 23280 heartbeater.cc:347] Connected to a master server at 127.11.60.62:36745
I20260811 03:12:40.456838 23280 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:40.457748 23280 heartbeater.cc:511] Master 127.11.60.62:36745 requested a full tablet report, sending...
I20260811 03:12:40.459306 22933 ts_manager.cc:194] Registered new tserver with Master: a127b3e446a14a3089ec8d22a3876b23 (127.11.60.2:46165)
I20260811 03:12:40.460145 22933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:53399
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:12:40.506582 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36745
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:12:40.847554 23287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:12:40.847831 23287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:12:40.847929 23287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:12:40.855082 23287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:12:40.855197 23287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:12:40.855309 23287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:12:40.865075 23287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:12:40.866357 23287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:12:40.867630 23287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:12:40.880785 23294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:40.880739 23297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:12:40.880823 23295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:12:40.881734 23287 server_base.cc:1034] running on GCE node
I20260811 03:12:40.882401 23287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:12:40.883345 23287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:12:40.884593 23287 hybrid_clock.cc:648] HybridClock initialized: now 1786417960884563 us; error 56 us; skew 500 ppm
I20260811 03:12:40.884905 23287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:12:40 dist-test-slave-tm1g krb5kdc[22884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417960, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:12:40.888170 23287 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:12:40.889737 23287 webserver.cc:460] Webserver started at http://127.11.60.3:40795/ using document root <none> and password file <none>
I20260811 03:12:40.890512 23287 fs_manager.cc:359] Metadata directory not provided
I20260811 03:12:40.890659 23287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:12:40.890959 23287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:12:40.893776 23287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "4a7483a5dd5543fcbf2764128951dc8a"
format_stamp: "Formatted at 2026-08-11 03:12:40 on dist-test-slave-tm1g"
server_key: "6a331da4e1e653b04b39bf73cef4c9a2"
server_key_iv: "28cca0d8693755b8d125af71e90a7e3a"
server_key_version: "encryptionkey@0"
I20260811 03:12:40.894605 23287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "4a7483a5dd5543fcbf2764128951dc8a"
format_stamp: "Formatted at 2026-08-11 03:12:40 on dist-test-slave-tm1g"
server_key: "6a331da4e1e653b04b39bf73cef4c9a2"
server_key_iv: "28cca0d8693755b8d125af71e90a7e3a"
server_key_version: "encryptionkey@0"
I20260811 03:12:40.899796 23287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 03:12:40.903445 23304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:40.904729 23287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 03:12:40.904917 23287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "4a7483a5dd5543fcbf2764128951dc8a"
format_stamp: "Formatted at 2026-08-11 03:12:40 on dist-test-slave-tm1g"
server_key: "6a331da4e1e653b04b39bf73cef4c9a2"
server_key_iv: "28cca0d8693755b8d125af71e90a7e3a"
server_key_version: "encryptionkey@0"
I20260811 03:12:40.905169 23287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:12:40.920181 23134 heartbeater.cc:503] Master 127.11.60.62:36745 was elected leader, sending a full tablet report...
I20260811 03:12:40.929960 23287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:12:40.934784 23287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:12:40.935101 23287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:12:40.936091 23287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:12:40.937453 23287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:12:40.937556 23287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:40.937706 23287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:12:40.937779 23287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:12:40.972268 23287 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:33999
I20260811 03:12:40.972353 23425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:33999 every 8 connection(s)
I20260811 03:12:40.973583 23287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:12:40.974759 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 23287
I20260811 03:12:40.974967 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:12:40.975533 11504 external_mini_cluster.cc:1428] Setting key 4019378ecbcc799a61139559e4dee388
Aug 11 03:12:40 dist-test-slave-tm1g krb5kdc[22884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417960, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:12:40.994155 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:40.976402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:52409 (local address 127.11.60.62:36745)
0811 03:12:40.976941 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:40.976948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:40.978040 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 03:12:40.980175 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:40.980180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:40.980272 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:40.980387 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:40.981849 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:40.982719 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:40.984259 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:40.984672 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:40.984805 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:40.988162 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:40.988167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:40.988170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:40.988201 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:40.991351 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:40.992066 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:40.992072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:40.992076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:40.992238 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:40.992688 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:40.992691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:40.992694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:40.992944 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:40.993064 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:40.993737 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 03:12:40.993894 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":270,"threads_started":1}
I20260811 03:12:40.995189 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:40.976985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:36745 (local address 127.11.60.3:52409)
0811 03:12:40.977869 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:40.977929 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:40.979914 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:40.980625 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:40.980633 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:40.981139 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:12:40.981682 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:40.981699 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:40.982881 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:40.982886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:40.984115 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:40.984126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:40.984904 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:40.984911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:40.985159 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:40.985786 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:12:40.985808 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:12:40.987949 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:12:40.991495 (+ 3546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:40.991505 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:40.991519 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:40.991937 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:40.992381 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:12:40.992385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:12:40.992387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:12:40.992578 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:12:40.993065 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:12:40.993070 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:12:40.993415 (+ 345us) client_negotiation.cc:769] Sending connection context
0811 03:12:40.993654 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 03:12:40.994052 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":321,"threads_started":1}
I20260811 03:12:40.996546 23426 heartbeater.cc:347] Connected to a master server at 127.11.60.62:36745
I20260811 03:12:40.996845 23426 heartbeater.cc:464] Registering TS with master...
I20260811 03:12:40.997553 23426 heartbeater.cc:511] Master 127.11.60.62:36745 requested a full tablet report, sending...
I20260811 03:12:40.998998 22933 ts_manager.cc:194] Registered new tserver with Master: 4a7483a5dd5543fcbf2764128951dc8a (127.11.60.3:33999)
I20260811 03:12:40.999792 22933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:52409
I20260811 03:12:41.003568 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:12:41.020745 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.008330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35894 (local address 127.11.60.62:36745)
0811 03:12:41.008568 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:41.008577 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:41.009103 (+ 526us) server_negotiation.cc:407] Connection header received
0811 03:12:41.009235 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:41.009239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:41.009299 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:41.009378 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:41.010986 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.011705 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.012949 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.013326 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.013414 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.015711 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:41.015717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:41.015720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:41.015750 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:41.018386 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:41.018995 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:41.018999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:41.019001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:41.019115 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:41.019430 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:41.019433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:41.019435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:41.019755 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:41.019935 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:41.020228 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 03:12:41.020387 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 03:12:41.024828 22933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:12:41.031924 22933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:12:41.066978 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.044750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58888 (local address 127.11.60.2:46165)
0811 03:12:41.056950 (+ 12200us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:41.056959 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:41.058038 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 03:12:41.058381 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:41.058390 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:41.058613 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:41.058823 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:41.060438 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.062139 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.064585 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.065768 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.065916 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.066142 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:41.066457 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 03:12:41.066620 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11999,"thread_start_us":12201,"threads_started":1}
I20260811 03:12:41.067028 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.044180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:46165 (local address 127.0.0.1:58888)
0811 03:12:41.057789 (+ 13609us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:41.057843 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:41.058063 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:41.059190 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:41.059201 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:41.059253 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:41.060268 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:41.060284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.062279 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.062285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:41.064383 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:41.064406 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.065968 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.065974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:41.066150 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.066221 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 03:12:41.066390 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 03:12:41.066591 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13471,"mutex_wait_us":1,"spinlock_wait_cycles":142208,"thread_start_us":13194,"threads_started":1}
I20260811 03:12:41.068756 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.044201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.11.60.3:33999)
0811 03:12:41.056577 (+ 12376us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:41.056590 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:41.058119 (+ 1529us) server_negotiation.cc:407] Connection header received
0811 03:12:41.058235 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:41.058243 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:41.058534 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:41.058780 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:41.060851 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.062447 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.065136 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.066590 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.067573 (+ 983us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.067782 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:41.068352 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 03:12:41.068499 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12142,"thread_start_us":12751,"threads_started":1}
I20260811 03:12:41.069027 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.058441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:36847 (local address 127.0.0.1:57566)
0811 03:12:41.060789 (+ 2348us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:41.060812 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:41.060937 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:41.061791 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:41.061796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:41.061820 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:41.062174 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:41.062188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.064285 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.064291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:41.066325 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:41.066342 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.068252 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.068262 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:41.068443 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.068485 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 03:12:41.068555 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 03:12:41.068671 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2237,"thread_start_us":2094,"threads_started":1}
I20260811 03:12:41.069478 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.044106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:33999 (local address 127.0.0.1:42240)
0811 03:12:41.057789 (+ 13683us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:41.057843 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:41.058049 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:41.059190 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:41.059201 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:41.059261 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:41.059990 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:41.060009 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.062599 (+ 2590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.062605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:41.064946 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:41.064965 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.067917 (+ 2952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.067924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:41.068117 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.068153 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 03:12:41.068979 (+ 826us) client_negotiation.cc:240] Negotiation successful
0811 03:12:41.069135 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13428,"mutex_wait_us":41,"spinlock_wait_cycles":27264,"thread_start_us":13239,"threads_started":1}
I20260811 03:12:41.071041 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.058603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.11.60.1:36847)
0811 03:12:41.060495 (+ 1892us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:41.060507 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:41.061063 (+ 556us) server_negotiation.cc:407] Connection header received
0811 03:12:41.061159 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:41.061167 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:41.061367 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:41.061546 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:41.062397 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.064128 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.066522 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.068027 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.070469 (+ 2442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.070642 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:41.070694 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 03:12:41.070810 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1684,"thread_start_us":1462,"threads_started":1}
I20260811 03:12:41.071151 23209 tablet_service.cc:1467] Processing CreateTablet for tablet 0d77b4ba6fa04437a0ed7c9d94be2639 (DEFAULT_TABLE table=test-table [id=2d4e2b610a2d492798be37eadf7bea72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:41.072726 23209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d77b4ba6fa04437a0ed7c9d94be2639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:41.073071 23353 tablet_service.cc:1467] Processing CreateTablet for tablet 0d77b4ba6fa04437a0ed7c9d94be2639 (DEFAULT_TABLE table=test-table [id=2d4e2b610a2d492798be37eadf7bea72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:41.073596 23066 tablet_service.cc:1467] Processing CreateTablet for tablet 0d77b4ba6fa04437a0ed7c9d94be2639 (DEFAULT_TABLE table=test-table [id=2d4e2b610a2d492798be37eadf7bea72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:12:41.074769 23066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d77b4ba6fa04437a0ed7c9d94be2639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:41.075301 23353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d77b4ba6fa04437a0ed7c9d94be2639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:12:41.085713 23447 tablet_bootstrap.cc:492] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: Bootstrap starting.
I20260811 03:12:41.089030 23447 tablet_bootstrap.cc:654] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:41.089952 23448 tablet_bootstrap.cc:492] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: Bootstrap starting.
I20260811 03:12:41.090499 23447 log.cc:824] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:41.090930 23449 tablet_bootstrap.cc:492] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: Bootstrap starting.
I20260811 03:12:41.094097 23448 tablet_bootstrap.cc:654] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:41.094412 23447 tablet_bootstrap.cc:492] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: No bootstrap required, opened a new log
I20260811 03:12:41.094511 23449 tablet_bootstrap.cc:654] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: Neither blocks nor log segments found. Creating new log.
I20260811 03:12:41.094717 23447 ts_tablet_manager.cc:1397] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 03:12:41.095475 23448 log.cc:824] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:41.095713 23449 log.cc:824] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: Log is configured to *not* fsync() on all Append() calls
I20260811 03:12:41.098282 23449 tablet_bootstrap.cc:492] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: No bootstrap required, opened a new log
I20260811 03:12:41.098563 23448 tablet_bootstrap.cc:492] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: No bootstrap required, opened a new log
I20260811 03:12:41.098603 23449 ts_tablet_manager.cc:1397] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 03:12:41.098814 23448 ts_tablet_manager.cc:1397] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260811 03:12:41.106000 23447 raft_consensus.cc:348] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a7483a5dd5543fcbf2764128951dc8a" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33999 } } peers { permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 } } peers { permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 } }
I20260811 03:12:41.106045 23449 raft_consensus.cc:348] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a7483a5dd5543fcbf2764128951dc8a" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33999 } } peers { permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 } } peers { permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 } }
I20260811 03:12:41.106473 23447 raft_consensus.cc:374] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:41.106494 23449 raft_consensus.cc:374] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:41.106575 23447 raft_consensus.cc:729] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a127b3e446a14a3089ec8d22a3876b23, State: Initialized, Role: FOLLOWER
I20260811 03:12:41.106582 23449 raft_consensus.cc:729] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039942daf13041d3b4f09c7e9d2dd436, State: Initialized, Role: FOLLOWER
I20260811 03:12:41.107201 23447 consensus_queue.cc:260] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a7483a5dd5543fcbf2764128951dc8a" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33999 } } peers { permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 } } peers { permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 } }
I20260811 03:12:41.107209 23449 consensus_queue.cc:260] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a7483a5dd5543fcbf2764128951dc8a" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33999 } } peers { permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 } } peers { permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 } }
I20260811 03:12:41.108481 23280 heartbeater.cc:503] Master 127.11.60.62:36745 was elected leader, sending a full tablet report...
I20260811 03:12:41.109297 23448 raft_consensus.cc:348] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a7483a5dd5543fcbf2764128951dc8a" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33999 } } peers { permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 } } peers { permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 } }
I20260811 03:12:41.109690 23448 raft_consensus.cc:374] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:12:41.109786 23448 raft_consensus.cc:729] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a7483a5dd5543fcbf2764128951dc8a, State: Initialized, Role: FOLLOWER
I20260811 03:12:41.109889 23449 ts_tablet_manager.cc:1428] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: Time spent starting tablet: real 0.011s user 0.007s sys 0.002s
I20260811 03:12:41.110443 23448 consensus_queue.cc:260] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a7483a5dd5543fcbf2764128951dc8a" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33999 } } peers { permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 } } peers { permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 } }
I20260811 03:12:41.112067 23447 ts_tablet_manager.cc:1428] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: Time spent starting tablet: real 0.017s user 0.013s sys 0.000s
I20260811 03:12:41.113018 23426 heartbeater.cc:503] Master 127.11.60.62:36745 was elected leader, sending a full tablet report...
I20260811 03:12:41.113608 23448 ts_tablet_manager.cc:1428] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
W20260811 03:12:41.142683 23135 tablet.cc:2369] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:12:41.183681 23281 tablet.cc:2369] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:12:41.225322 23427 tablet.cc:2369] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:12:41.283022 23455 raft_consensus.cc:482] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:12:41.283438 23455 raft_consensus.cc:504] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a7483a5dd5543fcbf2764128951dc8a" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33999 } } peers { permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 } } peers { permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 } }
I20260811 03:12:41.285426 23455 leader_election.cc:288] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a127b3e446a14a3089ec8d22a3876b23 (127.11.60.2:46165), 039942daf13041d3b4f09c7e9d2dd436 (127.11.60.1:36847)
I20260811 03:12:41.292285 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.285680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:46165 (local address 127.11.60.3:47251)
0811 03:12:41.285962 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:41.285987 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:41.286132 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:41.286636 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:41.286641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:41.286684 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:41.286910 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:41.286920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.288219 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.288223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:41.290168 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:41.290183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.291650 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.291654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:41.291786 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.291810 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:12:41.291885 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 03:12:41.291987 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260811 03:12:41.292332 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.285726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:47251 (local address 127.11.60.2:46165)
0811 03:12:41.285944 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:41.285950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:41.286068 (+ 118us) server_negotiation.cc:407] Connection header received
0811 03:12:41.286387 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:41.286396 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:41.286496 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:41.286590 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:41.287065 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.288077 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.290368 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.291494 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.291635 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.291731 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:41.291935 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 03:12:41.292072 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 03:12:41.293023 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.286146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:43811 (local address 127.11.60.1:36847)
0811 03:12:41.286371 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:41.286379 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:41.286889 (+ 510us) server_negotiation.cc:407] Connection header received
0811 03:12:41.287069 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:41.287075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:41.287157 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:41.287254 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:12:41.287739 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.288579 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.290769 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.292009 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.292137 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.292236 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:41.292639 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 03:12:41.292766 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 03:12:41.293047 23456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.285852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:36847 (local address 127.11.60.3:43811)
0811 03:12:41.286770 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0811 03:12:41.286787 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:12:41.286909 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:12:41.287273 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:12:41.287276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:12:41.287297 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:12:41.287581 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:41.287588 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.288730 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.288737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:41.290615 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:12:41.290626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.292230 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.292235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:12:41.292417 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.292450 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 03:12:41.292608 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 03:12:41.292729 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":551,"threads_started":1}
I20260811 03:12:41.293345 23229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d77b4ba6fa04437a0ed7c9d94be2639" candidate_uuid: "4a7483a5dd5543fcbf2764128951dc8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a127b3e446a14a3089ec8d22a3876b23" is_pre_election: true
I20260811 03:12:41.293875 23229 raft_consensus.cc:2454] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a7483a5dd5543fcbf2764128951dc8a in term 0.
I20260811 03:12:41.294009 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d77b4ba6fa04437a0ed7c9d94be2639" candidate_uuid: "4a7483a5dd5543fcbf2764128951dc8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039942daf13041d3b4f09c7e9d2dd436" is_pre_election: true
I20260811 03:12:41.294586 23087 raft_consensus.cc:2454] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a7483a5dd5543fcbf2764128951dc8a in term 0.
I20260811 03:12:41.294579 23306 leader_election.cc:302] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a7483a5dd5543fcbf2764128951dc8a, a127b3e446a14a3089ec8d22a3876b23; no voters:
I20260811 03:12:41.295017 23455 raft_consensus.cc:2781] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:12:41.295192 23455 raft_consensus.cc:482] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:12:41.295320 23455 raft_consensus.cc:3037] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:41.297545 23455 raft_consensus.cc:504] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a7483a5dd5543fcbf2764128951dc8a" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33999 } } peers { permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 } } peers { permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 } }
I20260811 03:12:41.298141 23455 leader_election.cc:288] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [CANDIDATE]: Term 1 election: Requested vote from peers a127b3e446a14a3089ec8d22a3876b23 (127.11.60.2:46165), 039942daf13041d3b4f09c7e9d2dd436 (127.11.60.1:36847)
I20260811 03:12:41.298655 23229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d77b4ba6fa04437a0ed7c9d94be2639" candidate_uuid: "4a7483a5dd5543fcbf2764128951dc8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a127b3e446a14a3089ec8d22a3876b23"
I20260811 03:12:41.298655 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d77b4ba6fa04437a0ed7c9d94be2639" candidate_uuid: "4a7483a5dd5543fcbf2764128951dc8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039942daf13041d3b4f09c7e9d2dd436"
I20260811 03:12:41.298862 23087 raft_consensus.cc:3037] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:41.298861 23229 raft_consensus.cc:3037] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:12:41.302083 23087 raft_consensus.cc:2454] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a7483a5dd5543fcbf2764128951dc8a in term 1.
I20260811 03:12:41.302083 23229 raft_consensus.cc:2454] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a7483a5dd5543fcbf2764128951dc8a in term 1.
I20260811 03:12:41.302507 23306 leader_election.cc:302] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a7483a5dd5543fcbf2764128951dc8a, a127b3e446a14a3089ec8d22a3876b23; no voters:
I20260811 03:12:41.302866 23455 raft_consensus.cc:2781] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:12:41.303341 23455 raft_consensus.cc:686] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 1 LEADER]: Becoming Leader. State: Replica: 4a7483a5dd5543fcbf2764128951dc8a, State: Running, Role: LEADER
I20260811 03:12:41.303872 23455 consensus_queue.cc:237] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a7483a5dd5543fcbf2764128951dc8a" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33999 } } peers { permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 } } peers { permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 } }
I20260811 03:12:41.308470 22931 catalog_manager.cc:5495] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a7483a5dd5543fcbf2764128951dc8a (127.11.60.3). New cstate: current_term: 1 leader_uuid: "4a7483a5dd5543fcbf2764128951dc8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a7483a5dd5543fcbf2764128951dc8a" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 } health_report { overall_health: UNKNOWN } } }
I20260811 03:12:41.353673 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.348701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.11.60.3:33999)
0811 03:12:41.348881 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:41.348885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:41.348902 (+ 17us) server_negotiation.cc:407] Connection header received
0811 03:12:41.349082 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:41.349087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:41.349174 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:41.349302 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:12:41.349662 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.350518 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.351794 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.352179 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.352270 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.352700 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:12:41.353224 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:12:41.353333 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:41.353409 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 03:12:41.353476 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 03:12:41.361740 23087 raft_consensus.cc:1261] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436 [term 1 FOLLOWER]: Refusing update from remote peer 4a7483a5dd5543fcbf2764128951dc8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:41.361745 23229 raft_consensus.cc:1261] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23 [term 1 FOLLOWER]: Refusing update from remote peer 4a7483a5dd5543fcbf2764128951dc8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:12:41.362828 23457 consensus_queue.cc:1035] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "039942daf13041d3b4f09c7e9d2dd436" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 36847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:41.363242 23455 consensus_queue.cc:1035] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a127b3e446a14a3089ec8d22a3876b23" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:12:41.374204 23461 mvcc.cc:205] Tried to move back new op lower bound from 7317167969728061440 to 7317167969504423936. Current Snapshot: MvccSnapshot[applied={T|T < 7317167969728061440}]
I20260811 03:12:41.388660 22931 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:41.388952 22931 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:12:41.392416 22931 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b3c8d22f1034908a72132eec376b109: Sending DeleteTablet for 3 replicas of tablet 0d77b4ba6fa04437a0ed7c9d94be2639
I20260811 03:12:41.393832 23066 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d77b4ba6fa04437a0ed7c9d94be2639 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:41 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:57566
I20260811 03:12:41.393985 23353 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d77b4ba6fa04437a0ed7c9d94be2639 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:41 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:42240
I20260811 03:12:41.394022 23209 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d77b4ba6fa04437a0ed7c9d94be2639 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:12:41 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:58888
I20260811 03:12:41.396634 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 22996
I20260811 03:12:41.406826 23473 tablet_replica.cc:330] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: stopping tablet replica
I20260811 03:12:41.406845 23472 tablet_replica.cc:330] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: stopping tablet replica
I20260811 03:12:41.407397 23473 raft_consensus.cc:2229] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:41.407521 23472 raft_consensus.cc:2229] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:12:41.408073 23473 raft_consensus.cc:2258] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:41.408460 23472 raft_consensus.cc:2258] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:41.409904 23471 tablet_replica.cc:330] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: stopping tablet replica
I20260811 03:12:41.410389 23471 raft_consensus.cc:2229] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:12:41.410914 23471 raft_consensus.cc:2258] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:12:41.411062 23473 ts_tablet_manager.cc:1905] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:41.413018 23471 ts_tablet_manager.cc:1905] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:41.413142 23472 ts_tablet_manager.cc:1905] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:12:41.418682 23473 ts_tablet_manager.cc:1918] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:41.418862 23473 log.cc:1196] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/0d77b4ba6fa04437a0ed7c9d94be2639
I20260811 03:12:41.419443 23473 ts_tablet_manager.cc:1939] T 0d77b4ba6fa04437a0ed7c9d94be2639 P a127b3e446a14a3089ec8d22a3876b23: Deleting consensus metadata
I20260811 03:12:41.420661 23471 ts_tablet_manager.cc:1918] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:41.420898 23471 log.cc:1196] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/0d77b4ba6fa04437a0ed7c9d94be2639
I20260811 03:12:41.421510 23471 ts_tablet_manager.cc:1939] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 039942daf13041d3b4f09c7e9d2dd436: Deleting consensus metadata
I20260811 03:12:41.421454 22917 catalog_manager.cc:4841] TS a127b3e446a14a3089ec8d22a3876b23 (127.11.60.2:46165): tablet 0d77b4ba6fa04437a0ed7c9d94be2639 (table test-table [id=2d4e2b610a2d492798be37eadf7bea72]) successfully deleted
I20260811 03:12:41.423460 22919 catalog_manager.cc:4841] TS 039942daf13041d3b4f09c7e9d2dd436 (127.11.60.1:36847): tablet 0d77b4ba6fa04437a0ed7c9d94be2639 (table test-table [id=2d4e2b610a2d492798be37eadf7bea72]) successfully deleted
I20260811 03:12:41.424177 23472 ts_tablet_manager.cc:1918] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:12:41.424357 23472 log.cc:1196] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/0d77b4ba6fa04437a0ed7c9d94be2639
I20260811 03:12:41.425004 23472 ts_tablet_manager.cc:1939] T 0d77b4ba6fa04437a0ed7c9d94be2639 P 4a7483a5dd5543fcbf2764128951dc8a: Deleting consensus metadata
I20260811 03:12:41.426497 22919 catalog_manager.cc:4841] TS 4a7483a5dd5543fcbf2764128951dc8a (127.11.60.3:33999): tablet 0d77b4ba6fa04437a0ed7c9d94be2639 (table test-table [id=2d4e2b610a2d492798be37eadf7bea72]) successfully deleted
Aug 11 03:12:41 dist-test-slave-tm1g krb5kdc[22884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:12:41.440129 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.398439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57574 (local address 127.11.60.1:36847)
0811 03:12:41.398695 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:41.398704 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:41.409063 (+ 10359us) server_negotiation.cc:407] Connection header received
0811 03:12:41.409320 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:41.409326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:41.409463 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:41.409601 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:41.418508 (+ 8907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.419777 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.421678 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.422385 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.422545 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.432529 (+ 9984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:41.432559 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:41.432578 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:41.432629 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:41.436685 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:41.437614 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:41.437624 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:41.437628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:41.437796 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:41.438212 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:41.438217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:41.438220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:41.438793 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:41.439141 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:41.439439 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 03:12:41.439698 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 03:12:41.440907 23129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:41.593259 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 22996
I20260811 03:12:41.614778 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 23141
Aug 11 03:12:41 dist-test-slave-tm1g krb5kdc[22884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:12:41.630666 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.615666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58896 (local address 127.11.60.2:46165)
0811 03:12:41.615870 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:41.615886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:41.615913 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:12:41.616079 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:41.616089 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:41.616167 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:41.616275 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:41.617547 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.618391 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.619689 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.620135 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.620289 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.624477 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:41.624492 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:41.624507 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:41.624540 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:41.627578 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:41.628287 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:41.628295 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:41.628297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:41.628455 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:41.628868 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:41.628885 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:41.628892 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:41.629289 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:41.629586 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:41.629932 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 03:12:41.630175 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 03:12:41.631418 23275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:41.723017 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 23141
I20260811 03:12:41.745911 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 23287
Aug 11 03:12:41 dist-test-slave-tm1g krb5kdc[22884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:12:41.764189 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:12:41.747472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.11.60.3:33999)
0811 03:12:41.747654 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 03:12:41.747660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:12:41.747681 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:12:41.747750 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:12:41.747757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:12:41.747845 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:12:41.747973 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:12:41.749231 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.750515 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.751931 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:12:41.752627 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:12:41.752772 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:12:41.757630 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:12:41.757645 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:12:41.757662 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:12:41.757712 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:12:41.761071 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:41.761978 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:41.761986 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:41.761988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:41.762133 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:12:41.762591 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:12:41.762595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:12:41.762596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:12:41.762992 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:12:41.763228 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 03:12:41.763651 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 03:12:41.763827 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 03:12:41.764959 23420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:41.892565 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 23287
I20260811 03:12:41.915205 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 22897
I20260811 03:12:41.916556 22965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:12:42.027932 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 22897
2026-08-11T03:12:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4691 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:12:42 dist-test-slave-tm1g krb5kdc[23482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:12:42 dist-test-slave-tm1g krb5kdc[23482](info): set up 2 sockets
Aug 11 03:12:42 dist-test-slave-tm1g krb5kdc[23482](info): commencing operation
krb5kdc: starting...
W20260811 03:12:43.223296 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:12:43 dist-test-slave-tm1g krb5kdc[23482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:12:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.60.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.60.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/rangeradmin_127.11.60.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/rangeradmin_127.11.60.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.60.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.60.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/rangerlookup_127.11.60.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/rangerlookup_127.11.60.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.60.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.60.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/HTTP_127.11.60.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/HTTP_127.11.60.20@KRBTEST.COM.keytab.
I20260811 03:12:43.633340 11504 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRu2AW0/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/postgres -l logfile start
2026-08-11 03:12:46.338 UTC [23513] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 03:12:46.338 UTC [23513] LOG: listening on IPv4 address "127.11.60.20", port 44637
2026-08-11 03:12:46.342 UTC [23513] LOG: listening on Unix socket "/tmp/.s.PGSQL.44637"
2026-08-11 03:12:46.367 UTC [23517] LOG: database system was shut down at 2026-08-11 03:12:44 UTC
2026-08-11 03:12:46.370 UTC [23513] LOG: database system is ready to accept connections
I20260811 03:12:47.410923 11504 mini_postgres.cc:96] Postgres bound to 44637
2026-08-11 03:12:47.435 UTC [23525] FATAL: database "slave" does not exist
127.11.60.20:44637 - accepting connections
I20260811 03:12:47.436343 11504 mini_ranger.cc:161] Starting Ranger...
I20260811 03:12:47.469165 11504 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 03:12:48.000748 11504 mini_ranger.cc:87] Created ranger Postgres database
I20260811 03:12:48.000916 11504 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-admin
2026-08-11 03:12:48,319 [I] DB FLAVOR :POSTGRES
2026-08-11 03:12:48,319 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 03:12:48,319 [I] Checking connection..
2026-08-11 03:12:48,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 03:12:48,573 [I] Checking connection passed.
2026-08-11 03:12:48,573 [I] --------- Verifying version history table ---------
2026-08-11 03:12:48,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 03:12:48,840 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 03:12:48,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 03:12:49,106 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 03:12:49,106 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 03:12:49,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 03:12:49.342 UTC [23597] WARNING: there is no transaction in progress
2026-08-11 03:12:49,349 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 03:12:49,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 03:12:49,600 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 03:12:49,600 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 03:12:49,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 03:12:49,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 03:12:50,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tm1g.c.gcp-upstream.internal','N') ;"
2026-08-11 03:12:50,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 03:12:50,611 [I] Table x_portal_user does not exist in database ranger
2026-08-11 03:12:50,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 03:12:50,864 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 03:12:50,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 03:12:51,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 03:12:51,382 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 03:12:51,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 03:12:52.421 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.438 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.457 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.474 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.492 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.511 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.570 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.577 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.594 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.605 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.616 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.630 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.641 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.648 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.656 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:52.664 UTC [23714] WARNING: there is no transaction in progress
2026-08-11 03:12:53,352 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 03:12:53,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 03:12:53,599 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 03:12:53,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 03:12:53,867 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 03:12:53,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 03:12:54,141 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 03:12:54,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 03:12:54,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 03:12:54,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tm1g.c.gcp-upstream.internal';"
2026-08-11 03:12:54,898 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 03:12:54,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 03:12:55,165 [I] DB_PATCHES have already been applied
I20260811 03:12:55.171804 11504 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRu2AW0/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 03:12:55.171891 11504 mini_ranger.cc:193] Using host: 127.11.60.20
I20260811 03:12:55.192920 11504 mini_ranger.cc:227] Ranger admin URL: http://127.11.60.20:36357
Aug 11, 2026 3:12:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 3:12:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 3:12:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 3:12:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 3:12:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 3:12:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 3:12:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 03:12:56.420542 11504 mini_ranger.cc:160] Time spent starting Ranger: real 8.984s user 0.002s sys 0.061s
Aug 11, 2026 3:13:17 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 3:13:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 11, 2026 3:13:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 3:13:17 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 3:13:18 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 03:13:18.817448 11504 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.60.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.60.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/rangerkms_127.11.60.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/rangerkms_127.11.60.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.60.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.60.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.60.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/HTTP_127.11.60.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.60.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/HTTP_127.11.60.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 03:13:19 dist-test-slave-tm1g krb5kdc[23482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417999, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 03:13:19.055964 11504 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 03:13:19.084002 11504 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 03:13:20.006534 11504 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 03:13:20.006721 11504 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-kms
I20260811 03:13:20.006842 11504 mini_ranger_kms.cc:227] Using postgres at 127.11.60.20:44637
2026-08-11 03:13:20,497 [I] DB FLAVOR :POSTGRES
2026-08-11 03:13:20,497 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 03:13:20,497 [I] Checking connection
2026-08-11 03:13:20,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 03:13:20,755 [I] connection success
2026-08-11 03:13:20,755 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 03:13:20,755 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 03:13:20,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 03:13:21,011 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 03:13:21,011 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 03:13:21,011 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 03:13:21,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:44637/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 03:13:21,274 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 03:13:21.811707 11504 mini_ranger_kms.cc:310] Created kms service
I20260811 03:13:21.989182 11504 mini_ranger_kms.cc:326] Created kudu user
I20260811 03:13:22.047784 11504 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 03:13:22 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417999, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:13:22.532606 11504 mini_ranger_kms.cc:384] Added ranger policy
I20260811 03:13:22.533048 11504 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 03:13:22.533129 11504 mini_ranger_kms.cc:240] Using host: 127.11.60.20
I20260811 03:13:22.565778 11504 mini_ranger_kms.cc:276] Ranger KMS PID: 23914
I20260811 03:13:22.566004 11504 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.60.20:46821
Aug 11, 2026 3:13:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 3:13:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 3:13:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 3:13:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 3:13:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 3:13:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 03:13:25.083680 11504 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.028s user 0.127s sys 0.408s
I20260811 03:13:25.084054 11504 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 03:13:25.084293 11504 mini_ranger_kms.cc:401] 127.11.60.20:46821/kms/v1/keys
Aug 11 03:13:25 dist-test-slave-tm1g krb5kdc[23482](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786418005, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.60.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 03:13:25 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418005, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.60.20@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
Aug 11 03:13:26 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418005, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.60.20@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
Aug 11 03:13:28 dist-test-slave-tm1g krb5kdc[23482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:13:28.503176 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:38541
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:38541
--ranger_config_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:46821/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:13:28.855882 23966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:13:28.856179 23966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:13:28.856268 23966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:13:28.863438 23966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:13:28.863549 23966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:13:28.863596 23966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:13:28.863659 23966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:13:28.863708 23966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:13:28.873828 23966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.60.20:46821/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:38541
--ranger_config_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:38541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:13:28.875191 23966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:13:28.876453 23966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:13:28.888595 23972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:13:28.888597 23974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:13:28.888785 23971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:13:28.889322 23966 server_base.cc:1034] running on GCE node
I20260811 03:13:28.890128 23966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:13:28.891614 23966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:13:28.892810 23966 hybrid_clock.cc:648] HybridClock initialized: now 1786418008892783 us; error 40 us; skew 500 ppm
I20260811 03:13:28.893178 23966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:13:28 dist-test-slave-tm1g krb5kdc[23482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:13:28.901429 23966 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:13:28.903002 23966 webserver.cc:460] Webserver started at http://127.11.60.62:42103/ using document root <none> and password file <none>
I20260811 03:13:28.903704 23966 fs_manager.cc:359] Metadata directory not provided
I20260811 03:13:28.903844 23966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:13:28.904155 23966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 03:13:28 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:13:28.964092 23966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "8594c657f45d4294b815e5f8064127d9"
format_stamp: "Formatted at 2026-08-11 03:13:28 on dist-test-slave-tm1g"
server_key: "5e0a20dc29eb6024157a468dda65b8ab"
server_key_iv: "e3af8e815ff8412a3706a836fb6065a9"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:28.965021 23966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "8594c657f45d4294b815e5f8064127d9"
format_stamp: "Formatted at 2026-08-11 03:13:28 on dist-test-slave-tm1g"
server_key: "5e0a20dc29eb6024157a468dda65b8ab"
server_key_iv: "e3af8e815ff8412a3706a836fb6065a9"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:28.970710 23966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 03:13:28.991285 23983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:13:28.993108 23966 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 03:13:28.993310 23966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "8594c657f45d4294b815e5f8064127d9"
format_stamp: "Formatted at 2026-08-11 03:13:28 on dist-test-slave-tm1g"
server_key: "5e0a20dc29eb6024157a468dda65b8ab"
server_key_iv: "e3af8e815ff8412a3706a836fb6065a9"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:28.993582 23966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:13:29.018836 23966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:13:29.023016 23966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:13:29.023269 23966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:13:29.041983 23966 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:38541
I20260811 03:13:29.042052 24042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:38541 every 8 connection(s)
I20260811 03:13:29.043469 23966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:13:29.047379 24043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:13:29.048853 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 23966
I20260811 03:13:29.049089 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:13:29.049955 11504 external_mini_cluster.cc:1428] Setting key 74200af603c14a0e3f506ca7f04f9281
I20260811 03:13:29.060514 24043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9: Bootstrap starting.
I20260811 03:13:29.064630 24043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9: Neither blocks nor log segments found. Creating new log.
Aug 11 03:13:29 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:13:29.065889 24043 log.cc:824] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9: Log is configured to *not* fsync() on all Append() calls
I20260811 03:13:29.069152 24043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9: No bootstrap required, opened a new log
I20260811 03:13:29.075227 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:29.052300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43936 (local address 127.11.60.62:38541)
0811 03:13:29.053253 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:29.053266 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:29.053322 (+ 56us) server_negotiation.cc:407] Connection header received
0811 03:13:29.055308 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:29.055333 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:29.055741 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:29.056087 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:13:29.057756 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:29.059326 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:29.061189 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:29.061924 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:29.062085 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:29.067206 (+ 5121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:13:29.067223 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:13:29.067245 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:13:29.067299 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:13:29.070645 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:29.071510 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:29.071520 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:29.071527 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:29.071722 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:29.072152 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:29.072155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:29.072157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:29.072748 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:13:29.072977 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:29.073548 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 03:13:29.073998 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":494,"threads_started":1}
I20260811 03:13:29.078025 24043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8594c657f45d4294b815e5f8064127d9" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 38541 } }
I20260811 03:13:29.078444 24043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:13:29.078570 24043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8594c657f45d4294b815e5f8064127d9, State: Initialized, Role: FOLLOWER
I20260811 03:13:29.079195 24043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8594c657f45d4294b815e5f8064127d9" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 38541 } }
I20260811 03:13:29.079443 24043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:13:29.079584 24043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:13:29.079753 24043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:13:29.082017 24043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8594c657f45d4294b815e5f8064127d9" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 38541 } }
I20260811 03:13:29.082617 24043 leader_election.cc:302] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8594c657f45d4294b815e5f8064127d9; no voters:
I20260811 03:13:29.083247 24043 leader_election.cc:288] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:13:29.083413 24050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:13:29.084156 24050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [term 1 LEADER]: Becoming Leader. State: Replica: 8594c657f45d4294b815e5f8064127d9, State: Running, Role: LEADER
I20260811 03:13:29.084797 24050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8594c657f45d4294b815e5f8064127d9" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 38541 } }
I20260811 03:13:29.085445 24043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:13:29.093204 24052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8594c657f45d4294b815e5f8064127d9. Latest consensus state: current_term: 1 leader_uuid: "8594c657f45d4294b815e5f8064127d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8594c657f45d4294b815e5f8064127d9" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 38541 } } }
I20260811 03:13:29.093370 24051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8594c657f45d4294b815e5f8064127d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8594c657f45d4294b815e5f8064127d9" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 38541 } } }
I20260811 03:13:29.093967 24051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [sys.catalog]: This master's current role is: LEADER
I20260811 03:13:29.093967 24052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9 [sys.catalog]: This master's current role is: LEADER
I20260811 03:13:29.096477 24060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:13:29.101200 24060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:13:29.102030 24043 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 03:13:29.120635 24060 catalog_manager.cc:1347] Generated new cluster ID: 4f25dcfd26c04e10ab1f6ce4e92d94f0
I20260811 03:13:29.120842 24060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:13:29.137189 24060 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:13:29.139441 24060 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:13:29.152174 24060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9: Generated new TSK 0
I20260811 03:13:29.153415 24060 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:13:29.970711 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:46821/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:38541
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:13:30 dist-test-slave-tm1g krb5kdc[23482](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786418010, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 03:13:30.467731 24092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:13:30.468227 24092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:13:30.468448 24092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:13:30.481009 24092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:13:30.481171 24092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:13:30.481428 24092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:13:30.493927 24092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.60.20:46821/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:38541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:13:30.495695 24092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:13:30.497069 24092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:13:30.530217 24104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:13:30.532143 24105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:13:30.543987 24107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:13:30.544720 24092 server_base.cc:1034] running on GCE node
I20260811 03:13:30.545783 24092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:13:30.548969 24092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:13:30.550324 24092 hybrid_clock.cc:648] HybridClock initialized: now 1786418010550143 us; error 128 us; skew 500 ppm
I20260811 03:13:30.550798 24092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:13:30 dist-test-slave-tm1g krb5kdc[23482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418010, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:13:30.555589 24092 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:13:30.562448 24092 webserver.cc:460] Webserver started at http://127.11.60.1:40221/ using document root <none> and password file <none>
I20260811 03:13:30.563519 24092 fs_manager.cc:359] Metadata directory not provided
I20260811 03:13:30.563674 24092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:13:30.564134 24092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 03:13:30 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418010, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:13:30.587958 24092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "e5c82a5280b443ac860d29580fe97136"
format_stamp: "Formatted at 2026-08-11 03:13:30 on dist-test-slave-tm1g"
server_key: "ec8b9d1a5b53bf2d37eca667a2fe3f11"
server_key_iv: "319b6f272d4085e500bdf6318dece66d"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:30.589267 24092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "e5c82a5280b443ac860d29580fe97136"
format_stamp: "Formatted at 2026-08-11 03:13:30 on dist-test-slave-tm1g"
server_key: "ec8b9d1a5b53bf2d37eca667a2fe3f11"
server_key_iv: "319b6f272d4085e500bdf6318dece66d"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:30.599835 24092 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.009s sys 0.000s
I20260811 03:13:30.631458 24114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:13:30.633648 24092 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.006s sys 0.003s
I20260811 03:13:30.633951 24092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "e5c82a5280b443ac860d29580fe97136"
format_stamp: "Formatted at 2026-08-11 03:13:30 on dist-test-slave-tm1g"
server_key: "ec8b9d1a5b53bf2d37eca667a2fe3f11"
server_key_iv: "319b6f272d4085e500bdf6318dece66d"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:30.634243 24092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:13:30.652563 24092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:13:30.657778 24092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:13:30.658124 24092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:13:30.665943 24092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:13:30.668476 24092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:13:30.668637 24092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 03:13:30.668859 24092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:13:30.669009 24092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 03:13:30 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418010, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:13:30.831732 24238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:34067 every 8 connection(s)
I20260811 03:13:30.831734 24092 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:34067
I20260811 03:13:30.833892 24092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:13:30.835554 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 24092
I20260811 03:13:30.835798 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:13:30.836764 11504 external_mini_cluster.cc:1428] Setting key c6a1b730717995071dc68c4d88d4153b
Aug 11 03:13:30 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418010, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
I20260811 03:13:30.888250 24242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:30.848231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:50763 (local address 127.11.60.62:38541)
0811 03:13:30.865049 (+ 16818us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:30.865057 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:30.865082 (+ 25us) server_negotiation.cc:407] Connection header received
0811 03:13:30.865154 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:30.865159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:30.865257 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:30.867710 (+ 2453us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:13:30.867996 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:30.869261 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:30.872122 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:30.872787 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:30.872912 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:30.878077 (+ 5165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:13:30.878084 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:13:30.878089 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:13:30.878164 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:13:30.881993 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:30.883009 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:30.883015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:30.883018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:30.883432 (+ 414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:30.884021 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:30.884026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:30.884029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:30.884422 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:13:30.884582 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:30.887521 (+ 2939us) server_negotiation.cc:299] Negotiation successful
0811 03:13:30.887798 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16672,"thread_start_us":17178,"threads_started":1}
I20260811 03:13:30.889285 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:30.847556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:38541 (local address 127.11.60.1:50763)
0811 03:13:30.855841 (+ 8285us) negotiation.cc:107] Waiting for socket to connect
0811 03:13:30.855909 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 03:13:30.860367 (+ 4458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:13:30.865923 (+ 5556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:13:30.865938 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:13:30.866674 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:13:30.867581 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:30.867609 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:30.869473 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:30.869479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:30.871893 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:30.871909 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:30.873255 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:30.873264 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:30.873576 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:30.874482 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:13:30.874514 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:13:30.877775 (+ 3261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:13:30.882173 (+ 4398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:13:30.882183 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:13:30.882203 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:13:30.882857 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:13:30.883589 (+ 732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:13:30.883594 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:13:30.883598 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:13:30.883851 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:13:30.884661 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:13:30.884669 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:13:30.885136 (+ 467us) client_negotiation.cc:769] Sending connection context
0811 03:13:30.885505 (+ 369us) client_negotiation.cc:240] Negotiation successful
0811 03:13:30.886180 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7922,"thread_start_us":7684,"threads_started":1}
I20260811 03:13:30.891573 24239 heartbeater.cc:347] Connected to a master server at 127.11.60.62:38541
I20260811 03:13:30.891908 24239 heartbeater.cc:464] Registering TS with master...
I20260811 03:13:30.892812 24239 heartbeater.cc:511] Master 127.11.60.62:38541 requested a full tablet report, sending...
I20260811 03:13:30.895677 24004 ts_manager.cc:194] Registered new tserver with Master: e5c82a5280b443ac860d29580fe97136 (127.11.60.1:34067)
I20260811 03:13:30.897771 24004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:50763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:13:30.935307 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:46821/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:38541
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:13:31 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418010, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
W20260811 03:13:31.458461 24245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:13:31.458781 24245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:13:31.458977 24245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:13:31.466389 24245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:13:31.466576 24245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:13:31.466789 24245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:13:31.477619 24245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.60.20:46821/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:38541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:13:31.479079 24245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:13:31.480413 24245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:13:31.496084 24260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:13:31.496578 24263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:13:31.497074 24261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:13:31.497419 24245 server_base.cc:1034] running on GCE node
I20260811 03:13:31.498183 24245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:13:31.499460 24245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:13:31.500752 24245 hybrid_clock.cc:648] HybridClock initialized: now 1786418011500668 us; error 57 us; skew 500 ppm
I20260811 03:13:31.501159 24245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:13:31 dist-test-slave-tm1g krb5kdc[23482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:13:31.505205 24245 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:13:31.507035 24245 webserver.cc:460] Webserver started at http://127.11.60.2:39089/ using document root <none> and password file <none>
I20260811 03:13:31.507783 24245 fs_manager.cc:359] Metadata directory not provided
I20260811 03:13:31.507915 24245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:13:31.508244 24245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 03:13:31 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:13:31.527571 24245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "1164a44ee56340949bd0590cce6731bc"
format_stamp: "Formatted at 2026-08-11 03:13:31 on dist-test-slave-tm1g"
server_key: "86a899bf15f211921b10a63d576807ff"
server_key_iv: "684b483e5a41a4cec2294b3eef9d71a5"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:31.528725 24245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "1164a44ee56340949bd0590cce6731bc"
format_stamp: "Formatted at 2026-08-11 03:13:31 on dist-test-slave-tm1g"
server_key: "86a899bf15f211921b10a63d576807ff"
server_key_iv: "684b483e5a41a4cec2294b3eef9d71a5"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:31.536643 24245 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260811 03:13:31.553422 24271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:13:31.555083 24245 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 03:13:31.555269 24245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "1164a44ee56340949bd0590cce6731bc"
format_stamp: "Formatted at 2026-08-11 03:13:31 on dist-test-slave-tm1g"
server_key: "86a899bf15f211921b10a63d576807ff"
server_key_iv: "684b483e5a41a4cec2294b3eef9d71a5"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:31.555531 24245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:13:31.575949 24245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:13:31.580246 24245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:13:31.580574 24245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:13:31.581719 24245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:13:31.583360 24245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:13:31.583462 24245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:13:31.583628 24245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:13:31.583710 24245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:13:31.628964 24245 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:40479
I20260811 03:13:31.628988 24393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:40479 every 8 connection(s)
I20260811 03:13:31.630645 24245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:13:31.631321 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 24245
I20260811 03:13:31.631495 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:13:31.632040 11504 external_mini_cluster.cc:1428] Setting key ac82b3953fd83bb8313a8c177d422dd5
Aug 11 03:13:31 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:13:31.662873 24397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:31.635862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:45509 (local address 127.11.60.62:38541)
0811 03:13:31.638846 (+ 2984us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:31.638855 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:31.642385 (+ 3530us) server_negotiation.cc:407] Connection header received
0811 03:13:31.642835 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:31.642841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:31.642927 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:31.643034 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:13:31.645150 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:31.646376 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:31.648732 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:31.649358 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:31.649538 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:31.654465 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:13:31.654472 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:13:31.654477 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:13:31.654519 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:13:31.658469 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:31.659640 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:31.659646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:31.659650 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:31.659830 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:31.660447 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:31.660452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:31.660455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:31.660905 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:13:31.661050 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:31.662135 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0811 03:13:31.662448 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2818,"thread_start_us":2689,"threads_started":1}
I20260811 03:13:31.664389 24396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:31.635310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:38541 (local address 127.11.60.2:45509)
0811 03:13:31.639520 (+ 4210us) negotiation.cc:107] Waiting for socket to connect
0811 03:13:31.639571 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 03:13:31.642492 (+ 2921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:13:31.643334 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:13:31.643349 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:13:31.644052 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:13:31.644947 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:31.644976 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:31.646686 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:31.646694 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:31.648578 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:31.648590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:31.649665 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:31.649675 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:31.650052 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:31.650960 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:13:31.650995 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:13:31.654169 (+ 3174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:13:31.658648 (+ 4479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:13:31.658662 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:13:31.658685 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:13:31.659508 (+ 823us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:13:31.659961 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:13:31.659969 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:13:31.659975 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:13:31.660303 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:13:31.661050 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:13:31.661060 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:13:31.661639 (+ 579us) client_negotiation.cc:769] Sending connection context
0811 03:13:31.662011 (+ 372us) client_negotiation.cc:240] Negotiation successful
0811 03:13:31.662731 (+ 720us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3865,"thread_start_us":3594,"threads_started":1}
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
I20260811 03:13:31.666353 24394 heartbeater.cc:347] Connected to a master server at 127.11.60.62:38541
I20260811 03:13:31.666723 24394 heartbeater.cc:464] Registering TS with master...
I20260811 03:13:31.667650 24394 heartbeater.cc:511] Master 127.11.60.62:38541 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
I20260811 03:13:31.669489 24004 ts_manager.cc:194] Registered new tserver with Master: 1164a44ee56340949bd0590cce6731bc (127.11.60.2:40479)
I20260811 03:13:31.670859 24004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:45509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:13:31.705950 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:46821/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:38541
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:13:31.901865 24239 heartbeater.cc:503] Master 127.11.60.62:38541 was elected leader, sending a full tablet report...
W20260811 03:13:32.091588 24401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:13:32.091885 24401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:13:32.092098 24401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:13:32.100359 24401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:13:32.100543 24401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:13:32.100716 24401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:13:32.111073 24401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.60.20:46821/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:38541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:13:32.112506 24401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:13:32.113880 24401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:13:32.127766 24406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:13:32.127934 24407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:13:32.127976 24411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:13:32.129166 24401 server_base.cc:1034] running on GCE node
I20260811 03:13:32.129803 24401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:13:32.130856 24401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:13:32.132084 24401 hybrid_clock.cc:648] HybridClock initialized: now 1786418012132020 us; error 61 us; skew 500 ppm
I20260811 03:13:32.132400 24401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:13:32 dist-test-slave-tm1g krb5kdc[23482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418012, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:13:32.135638 24401 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:13:32.137084 24401 webserver.cc:460] Webserver started at http://127.11.60.3:37703/ using document root <none> and password file <none>
I20260811 03:13:32.137795 24401 fs_manager.cc:359] Metadata directory not provided
I20260811 03:13:32.137928 24401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:13:32.138191 24401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 03:13:32 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418012, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:13:32.156361 24401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "66dc0c726c974e45be190efd73aa93f0"
format_stamp: "Formatted at 2026-08-11 03:13:32 on dist-test-slave-tm1g"
server_key: "c99233e6bfbb14e114b1d4455a0523f6"
server_key_iv: "c32f2104953b47ecc0aa5f49b2292989"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:32.157522 24401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "66dc0c726c974e45be190efd73aa93f0"
format_stamp: "Formatted at 2026-08-11 03:13:32 on dist-test-slave-tm1g"
server_key: "c99233e6bfbb14e114b1d4455a0523f6"
server_key_iv: "c32f2104953b47ecc0aa5f49b2292989"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:32.164876 24401 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 03:13:32.182686 24418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:13:32.184320 24401 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 03:13:32.184520 24401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "66dc0c726c974e45be190efd73aa93f0"
format_stamp: "Formatted at 2026-08-11 03:13:32 on dist-test-slave-tm1g"
server_key: "c99233e6bfbb14e114b1d4455a0523f6"
server_key_iv: "c32f2104953b47ecc0aa5f49b2292989"
server_key_version: "kuduclusterkey@0"
I20260811 03:13:32.184782 24401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:13:32.201392 24401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:13:32.205161 24401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:13:32.205472 24401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:13:32.206480 24401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:13:32.207883 24401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:13:32.208000 24401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:13:32.208165 24401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:13:32.208248 24401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:13:32.248683 24401 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:46141
I20260811 03:13:32.248697 24538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:46141 every 8 connection(s)
I20260811 03:13:32.250211 24401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:13:32.255961 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 24401
I20260811 03:13:32.256153 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:13:32.256592 11504 external_mini_cluster.cc:1428] Setting key e3b819cc95913ecb3e9bfe6f702f09dc
Aug 11 03:13:32 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418012, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:13:32.271065 24541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.252715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:47535 (local address 127.11.60.62:38541)
0811 03:13:32.253277 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:32.253282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:32.254152 (+ 870us) server_negotiation.cc:407] Connection header received
0811 03:13:32.256196 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:32.256203 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:32.256304 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:32.256424 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:13:32.258243 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.259062 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.260891 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.261331 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.261466 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.265044 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:13:32.265050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:13:32.265054 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:13:32.265088 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:13:32.267896 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:32.268641 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:32.268648 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:32.268652 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:32.268797 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:32.269245 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:32.269252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:32.269255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:32.269553 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:13:32.269724 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:32.270417 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 03:13:32.270653 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":313,"threads_started":1}
I20260811 03:13:32.272215 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.253225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:38541 (local address 127.11.60.3:47535)
0811 03:13:32.253993 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 03:13:32.254030 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 03:13:32.255890 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:13:32.256659 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:13:32.256685 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:13:32.257149 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:13:32.258082 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.258103 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.259226 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.259232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.260733 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.260747 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.261557 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.261566 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.261871 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.262598 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:13:32.262621 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:13:32.264813 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:13:32.268065 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:13:32.268078 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:13:32.268091 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:13:32.268523 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:13:32.268914 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:13:32.268919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:13:32.268923 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:13:32.269133 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:13:32.269688 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:13:32.269696 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:13:32.270017 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 03:13:32.270286 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 03:13:32.270841 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":333,"threads_started":1}
I20260811 03:13:32.273608 24539 heartbeater.cc:347] Connected to a master server at 127.11.60.62:38541
I20260811 03:13:32.273890 24539 heartbeater.cc:464] Registering TS with master...
I20260811 03:13:32.274642 24539 heartbeater.cc:511] Master 127.11.60.62:38541 requested a full tablet report, sending...
I20260811 03:13:32.275964 24004 ts_manager.cc:194] Registered new tserver with Master: 66dc0c726c974e45be190efd73aa93f0 (127.11.60.3:46141)
I20260811 03:13:32.276907 24004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:47535
I20260811 03:13:32.284677 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:13:32.309237 24541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.289726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.11.60.62:38541)
0811 03:13:32.289899 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:32.289903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:32.290767 (+ 864us) server_negotiation.cc:407] Connection header received
0811 03:13:32.290959 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:32.290964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:32.291049 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:32.291149 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:13:32.292897 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.293649 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.295283 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.295724 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.295864 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.303164 (+ 7300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:13:32.303170 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:13:32.303173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:13:32.303209 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:13:32.305939 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:32.306939 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:32.306943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:32.306945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:32.307076 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:32.307581 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:32.307585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:32.307587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:32.307870 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:13:32.308212 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:32.308610 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 03:13:32.308909 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 03:13:32.313346 24004 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:13:32.321087 24004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:13:32.349925 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.339273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.11.60.1:34067)
0811 03:13:32.341188 (+ 1915us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:32.341198 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:32.341773 (+ 575us) server_negotiation.cc:407] Connection header received
0811 03:13:32.341880 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:32.341887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:32.342095 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:32.342282 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:13:32.342932 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.343982 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.346722 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.348294 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.348440 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.348712 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:32.349367 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 03:13:32.349547 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1743,"thread_start_us":1585,"threads_started":1}
I20260811 03:13:32.350207 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.339605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34067 (local address 127.0.0.1:55022)
0811 03:13:32.341513 (+ 1908us) negotiation.cc:107] Waiting for socket to connect
0811 03:13:32.341531 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:13:32.341654 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:13:32.342287 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:13:32.342292 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:13:32.342342 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:13:32.342738 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.342751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.344128 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.344136 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.346516 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.346529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.348875 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.348881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.349139 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.349183 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 03:13:32.349308 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 03:13:32.349536 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1753,"mutex_wait_us":54,"thread_start_us":1625,"threads_started":1}
I20260811 03:13:32.350526 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.334966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.11.60.3:46141)
0811 03:13:32.335674 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:32.335687 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:32.338980 (+ 3293us) server_negotiation.cc:407] Connection header received
0811 03:13:32.339277 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:32.339284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:32.339546 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:32.339749 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:13:32.341093 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.342821 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.346651 (+ 3830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.348610 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.348746 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.348993 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:32.349626 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 03:13:32.349837 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":316,"threads_started":1}
I20260811 03:13:32.350781 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.334662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40456 (local address 127.11.60.2:40479)
0811 03:13:32.339096 (+ 4434us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:32.339106 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:32.339136 (+ 30us) server_negotiation.cc:407] Connection header received
0811 03:13:32.339299 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:32.339306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:32.339597 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:32.340953 (+ 1356us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:13:32.341230 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.342762 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.346691 (+ 3929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.348269 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.348416 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.348634 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:32.350201 (+ 1567us) server_negotiation.cc:299] Negotiation successful
0811 03:13:32.350419 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4254,"thread_start_us":4122,"threads_started":1}
I20260811 03:13:32.350821 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.334484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:40479 (local address 127.0.0.1:40456)
0811 03:13:32.338066 (+ 3582us) negotiation.cc:107] Waiting for socket to connect
0811 03:13:32.338200 (+ 134us) client_negotiation.cc:174] Beginning negotiation
0811 03:13:32.338462 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:13:32.339906 (+ 1444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:13:32.339918 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:13:32.339968 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:13:32.340712 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.340730 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.342893 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.342901 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.346481 (+ 3580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.346500 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.348779 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.348785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.349024 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.349123 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 03:13:32.349309 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 03:13:32.349523 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3245,"mutex_wait_us":234,"thread_start_us":4037,"threads_started":1}
I20260811 03:13:32.351348 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.334651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:46141 (local address 127.0.0.1:43186)
0811 03:13:32.338854 (+ 4203us) negotiation.cc:107] Waiting for socket to connect
0811 03:13:32.338874 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:13:32.339084 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:13:32.340233 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:13:32.340241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:13:32.340283 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:13:32.340831 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.340846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.342959 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.342963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.346488 (+ 3525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.346500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.349182 (+ 2682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.349188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.349420 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.349452 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 03:13:32.350741 (+ 1289us) client_negotiation.cc:240] Negotiation successful
0811 03:13:32.351009 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4069,"mutex_wait_us":193,"thread_start_us":2900,"threads_started":1}
I20260811 03:13:32.354779 24323 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbb0cd7041045da8cd6e91f5b51ac0a (DEFAULT_TABLE table=test-table [id=6bb6c2ce720c404d9360f5362f9e6668]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:13:32.354827 24166 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbb0cd7041045da8cd6e91f5b51ac0a (DEFAULT_TABLE table=test-table [id=6bb6c2ce720c404d9360f5362f9e6668]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:13:32.354835 24469 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbb0cd7041045da8cd6e91f5b51ac0a (DEFAULT_TABLE table=test-table [id=6bb6c2ce720c404d9360f5362f9e6668]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:13:32.356135 24323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbb0cd7041045da8cd6e91f5b51ac0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:13:32.356298 24469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbb0cd7041045da8cd6e91f5b51ac0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:13:32.356314 24166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbb0cd7041045da8cd6e91f5b51ac0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:13:32.369010 24558 tablet_bootstrap.cc:492] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: Bootstrap starting.
I20260811 03:13:32.371246 24559 tablet_bootstrap.cc:492] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: Bootstrap starting.
I20260811 03:13:32.373338 24560 tablet_bootstrap.cc:492] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: Bootstrap starting.
I20260811 03:13:32.375663 24558 tablet_bootstrap.cc:654] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: Neither blocks nor log segments found. Creating new log.
I20260811 03:13:32.375741 24559 tablet_bootstrap.cc:654] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: Neither blocks nor log segments found. Creating new log.
I20260811 03:13:32.376638 24560 tablet_bootstrap.cc:654] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: Neither blocks nor log segments found. Creating new log.
I20260811 03:13:32.377409 24559 log.cc:824] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: Log is configured to *not* fsync() on all Append() calls
I20260811 03:13:32.377413 24558 log.cc:824] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: Log is configured to *not* fsync() on all Append() calls
I20260811 03:13:32.377748 24560 log.cc:824] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: Log is configured to *not* fsync() on all Append() calls
I20260811 03:13:32.381170 24560 tablet_bootstrap.cc:492] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: No bootstrap required, opened a new log
I20260811 03:13:32.381268 24559 tablet_bootstrap.cc:492] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: No bootstrap required, opened a new log
I20260811 03:13:32.381471 24560 ts_tablet_manager.cc:1397] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 03:13:32.381546 24559 ts_tablet_manager.cc:1397] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260811 03:13:32.386674 24558 tablet_bootstrap.cc:492] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: No bootstrap required, opened a new log
I20260811 03:13:32.387009 24558 ts_tablet_manager.cc:1397] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.003s
I20260811 03:13:32.390018 24559 raft_consensus.cc:348] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 } } peers { permanent_uuid: "66dc0c726c974e45be190efd73aa93f0" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46141 } } peers { permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 } }
I20260811 03:13:32.390468 24559 raft_consensus.cc:374] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:13:32.390597 24559 raft_consensus.cc:729] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c82a5280b443ac860d29580fe97136, State: Initialized, Role: FOLLOWER
I20260811 03:13:32.391301 24559 consensus_queue.cc:260] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 } } peers { permanent_uuid: "66dc0c726c974e45be190efd73aa93f0" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46141 } } peers { permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 } }
I20260811 03:13:32.392232 24560 raft_consensus.cc:348] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 } } peers { permanent_uuid: "66dc0c726c974e45be190efd73aa93f0" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46141 } } peers { permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 } }
I20260811 03:13:32.392714 24560 raft_consensus.cc:374] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:13:32.392892 24560 raft_consensus.cc:729] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66dc0c726c974e45be190efd73aa93f0, State: Initialized, Role: FOLLOWER
I20260811 03:13:32.393534 24560 consensus_queue.cc:260] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 } } peers { permanent_uuid: "66dc0c726c974e45be190efd73aa93f0" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46141 } } peers { permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 } }
I20260811 03:13:32.394977 24539 heartbeater.cc:503] Master 127.11.60.62:38541 was elected leader, sending a full tablet report...
I20260811 03:13:32.395699 24560 ts_tablet_manager.cc:1428] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260811 03:13:32.397943 24559 ts_tablet_manager.cc:1428] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
I20260811 03:13:32.399564 24558 raft_consensus.cc:348] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 } } peers { permanent_uuid: "66dc0c726c974e45be190efd73aa93f0" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46141 } } peers { permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 } }
I20260811 03:13:32.400053 24558 raft_consensus.cc:374] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:13:32.400171 24558 raft_consensus.cc:729] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1164a44ee56340949bd0590cce6731bc, State: Initialized, Role: FOLLOWER
I20260811 03:13:32.400789 24558 consensus_queue.cc:260] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 } } peers { permanent_uuid: "66dc0c726c974e45be190efd73aa93f0" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46141 } } peers { permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 } }
I20260811 03:13:32.401979 24394 heartbeater.cc:503] Master 127.11.60.62:38541 was elected leader, sending a full tablet report...
I20260811 03:13:32.402729 24558 ts_tablet_manager.cc:1428] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
W20260811 03:13:32.501957 24540 tablet.cc:2369] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:13:32.597615 24240 tablet.cc:2369] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:13:32.634564 24395 tablet.cc:2369] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:13:32.663880 24565 raft_consensus.cc:482] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:13:32.664213 24565 raft_consensus.cc:504] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 } } peers { permanent_uuid: "66dc0c726c974e45be190efd73aa93f0" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46141 } } peers { permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 } }
I20260811 03:13:32.666007 24565 leader_election.cc:288] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1164a44ee56340949bd0590cce6731bc (127.11.60.2:40479), e5c82a5280b443ac860d29580fe97136 (127.11.60.1:34067)
I20260811 03:13:32.672043 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.666814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:39257 (local address 127.11.60.2:40479)
0811 03:13:32.667059 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:32.667065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:32.667086 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:13:32.667181 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:32.667186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:32.667286 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:32.667391 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:13:32.667928 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.668669 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.670346 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.671148 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.671299 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.671393 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:32.671656 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 03:13:32.671786 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 03:13:32.672048 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.666597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:40479 (local address 127.11.60.3:39257)
0811 03:13:32.666792 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 03:13:32.666812 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:13:32.666961 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:13:32.667448 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:13:32.667453 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:13:32.667498 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:13:32.667780 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.667790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.668812 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.668821 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.670155 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.670167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.671305 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.671311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.671492 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.671518 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 03:13:32.671625 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 03:13:32.671753 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 03:13:32.673064 24568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.667125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34067 (local address 127.11.60.3:55677)
0811 03:13:32.667972 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 03:13:32.667985 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:13:32.668127 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:13:32.668435 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:13:32.668439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:13:32.668466 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:13:32.668791 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.668800 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.669841 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.669845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.671201 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:13:32.671212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.672399 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.672407 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:13:32.672599 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.672630 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 03:13:32.672731 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 03:13:32.672836 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":539,"threads_started":1}
I20260811 03:13:32.673225 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.667425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:55677 (local address 127.11.60.1:34067)
0811 03:13:32.667657 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:32.667664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:32.668075 (+ 411us) server_negotiation.cc:407] Connection header received
0811 03:13:32.668259 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:32.668263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:32.668322 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:32.668413 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:13:32.668944 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.669681 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.671353 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.672170 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.672313 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.672400 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:32.672809 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 03:13:32.672959 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 03:13:32.673334 24345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbb0cd7041045da8cd6e91f5b51ac0a" candidate_uuid: "66dc0c726c974e45be190efd73aa93f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1164a44ee56340949bd0590cce6731bc" is_pre_election: true
I20260811 03:13:32.673902 24345 raft_consensus.cc:2454] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66dc0c726c974e45be190efd73aa93f0 in term 0.
I20260811 03:13:32.674180 24188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbb0cd7041045da8cd6e91f5b51ac0a" candidate_uuid: "66dc0c726c974e45be190efd73aa93f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c82a5280b443ac860d29580fe97136" is_pre_election: true
I20260811 03:13:32.674602 24421 leader_election.cc:302] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1164a44ee56340949bd0590cce6731bc, 66dc0c726c974e45be190efd73aa93f0; no voters:
I20260811 03:13:32.674778 24188 raft_consensus.cc:2454] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66dc0c726c974e45be190efd73aa93f0 in term 0.
I20260811 03:13:32.674974 24565 raft_consensus.cc:2781] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:13:32.675123 24565 raft_consensus.cc:482] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:13:32.675244 24565 raft_consensus.cc:3037] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:13:32.677597 24565 raft_consensus.cc:504] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 } } peers { permanent_uuid: "66dc0c726c974e45be190efd73aa93f0" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46141 } } peers { permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 } }
I20260811 03:13:32.678184 24565 leader_election.cc:288] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [CANDIDATE]: Term 1 election: Requested vote from peers 1164a44ee56340949bd0590cce6731bc (127.11.60.2:40479), e5c82a5280b443ac860d29580fe97136 (127.11.60.1:34067)
I20260811 03:13:32.678617 24345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbb0cd7041045da8cd6e91f5b51ac0a" candidate_uuid: "66dc0c726c974e45be190efd73aa93f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1164a44ee56340949bd0590cce6731bc"
I20260811 03:13:32.678709 24188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbb0cd7041045da8cd6e91f5b51ac0a" candidate_uuid: "66dc0c726c974e45be190efd73aa93f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c82a5280b443ac860d29580fe97136"
I20260811 03:13:32.678819 24345 raft_consensus.cc:3037] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:13:32.678937 24188 raft_consensus.cc:3037] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:13:32.681044 24345 raft_consensus.cc:2454] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66dc0c726c974e45be190efd73aa93f0 in term 1.
I20260811 03:13:32.681090 24188 raft_consensus.cc:2454] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66dc0c726c974e45be190efd73aa93f0 in term 1.
I20260811 03:13:32.681423 24421 leader_election.cc:302] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1164a44ee56340949bd0590cce6731bc, 66dc0c726c974e45be190efd73aa93f0; no voters:
I20260811 03:13:32.681762 24565 raft_consensus.cc:2781] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:13:32.682152 24565 raft_consensus.cc:686] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 1 LEADER]: Becoming Leader. State: Replica: 66dc0c726c974e45be190efd73aa93f0, State: Running, Role: LEADER
I20260811 03:13:32.682662 24565 consensus_queue.cc:237] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 } } peers { permanent_uuid: "66dc0c726c974e45be190efd73aa93f0" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46141 } } peers { permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 } }
I20260811 03:13:32.687647 24003 catalog_manager.cc:5495] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66dc0c726c974e45be190efd73aa93f0 (127.11.60.3). New cstate: current_term: 1 leader_uuid: "66dc0c726c974e45be190efd73aa93f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66dc0c726c974e45be190efd73aa93f0" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 46141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 } health_report { overall_health: UNKNOWN } } }
I20260811 03:13:32.720213 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.714835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.11.60.3:46141)
0811 03:13:32.715042 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:32.715047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:32.715065 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:13:32.715152 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:32.715156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:32.715242 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:32.715366 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:13:32.715715 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.716458 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.717823 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.718220 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.718357 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.718935 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:13:32.719536 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:13:32.719649 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:32.719762 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 03:13:32.719831 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 03:13:32.729185 24345 raft_consensus.cc:1261] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc [term 1 FOLLOWER]: Refusing update from remote peer 66dc0c726c974e45be190efd73aa93f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:13:32.729195 24188 raft_consensus.cc:1261] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136 [term 1 FOLLOWER]: Refusing update from remote peer 66dc0c726c974e45be190efd73aa93f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:13:32.730135 24569 consensus_queue.cc:1035] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c82a5280b443ac860d29580fe97136" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:13:32.730563 24565 consensus_queue.cc:1035] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1164a44ee56340949bd0590cce6731bc" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 40479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:13:32.742295 24573 mvcc.cc:205] Tried to move back new op lower bound from 7317168180129144832 to 7317168179952189440. Current Snapshot: MvccSnapshot[applied={T|T < 7317168180129144832}]
I20260811 03:13:32.758198 24004 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:13:32.758548 24004 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:13:32.762436 24004 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8594c657f45d4294b815e5f8064127d9: Sending DeleteTablet for 3 replicas of tablet 7bbb0cd7041045da8cd6e91f5b51ac0a
I20260811 03:13:32.763638 24323 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bbb0cd7041045da8cd6e91f5b51ac0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:13:32 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:40456
I20260811 03:13:32.764429 24166 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bbb0cd7041045da8cd6e91f5b51ac0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:13:32 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:55022
I20260811 03:13:32.764843 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 24092
I20260811 03:13:32.765586 24469 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bbb0cd7041045da8cd6e91f5b51ac0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:13:32 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:43186
I20260811 03:13:32.767325 24583 tablet_replica.cc:330] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: stopping tablet replica
I20260811 03:13:32.767942 24583 raft_consensus.cc:2229] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:13:32.768653 24583 raft_consensus.cc:2258] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:13:32.771570 24583 ts_tablet_manager.cc:1905] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:13:32.775126 24584 tablet_replica.cc:330] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: stopping tablet replica
I20260811 03:13:32.775758 24584 raft_consensus.cc:2229] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:13:32.776376 24584 raft_consensus.cc:2258] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:13:32.779237 24584 ts_tablet_manager.cc:1905] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:13:32.781759 24583 ts_tablet_manager.cc:1918] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:13:32.782063 24583 log.cc:1196] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/7bbb0cd7041045da8cd6e91f5b51ac0a
I20260811 03:13:32.782855 24583 ts_tablet_manager.cc:1939] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 1164a44ee56340949bd0590cce6731bc: Deleting consensus metadata
I20260811 03:13:32.785648 23986 catalog_manager.cc:4841] TS 1164a44ee56340949bd0590cce6731bc (127.11.60.2:40479): tablet 7bbb0cd7041045da8cd6e91f5b51ac0a (table test-table [id=6bb6c2ce720c404d9360f5362f9e6668]) successfully deleted
I20260811 03:13:32.787281 24584 ts_tablet_manager.cc:1918] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:13:32.787575 24584 log.cc:1196] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/7bbb0cd7041045da8cd6e91f5b51ac0a
I20260811 03:13:32.788200 24584 ts_tablet_manager.cc:1939] T 7bbb0cd7041045da8cd6e91f5b51ac0a P e5c82a5280b443ac860d29580fe97136: Deleting consensus metadata
I20260811 03:13:32.788978 24585 tablet_replica.cc:330] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: stopping tablet replica
I20260811 03:13:32.789395 24585 raft_consensus.cc:2229] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:13:32.790198 24585 raft_consensus.cc:2258] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:13:32.790381 23986 catalog_manager.cc:4841] TS e5c82a5280b443ac860d29580fe97136 (127.11.60.1:34067): tablet 7bbb0cd7041045da8cd6e91f5b51ac0a (table test-table [id=6bb6c2ce720c404d9360f5362f9e6668]) successfully deleted
I20260811 03:13:32.792531 24585 ts_tablet_manager.cc:1905] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 03:13:32 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:13:32.799345 24585 ts_tablet_manager.cc:1918] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:13:32.799561 24585 log.cc:1196] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/7bbb0cd7041045da8cd6e91f5b51ac0a
I20260811 03:13:32.800213 24585 ts_tablet_manager.cc:1939] T 7bbb0cd7041045da8cd6e91f5b51ac0a P 66dc0c726c974e45be190efd73aa93f0: Deleting consensus metadata
I20260811 03:13:32.802354 23988 catalog_manager.cc:4841] TS 66dc0c726c974e45be190efd73aa93f0 (127.11.60.3:46141): tablet 7bbb0cd7041045da8cd6e91f5b51ac0a (table test-table [id=6bb6c2ce720c404d9360f5362f9e6668]) successfully deleted
I20260811 03:13:32.806066 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.766251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55030 (local address 127.11.60.1:34067)
0811 03:13:32.766490 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:32.766499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:32.781191 (+ 14692us) server_negotiation.cc:407] Connection header received
0811 03:13:32.781723 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:32.781729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:32.781828 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:32.781901 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:13:32.788417 (+ 6516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.789560 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.792005 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.792687 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.792831 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.798634 (+ 5803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:13:32.798654 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:13:32.798673 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:13:32.798720 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:13:32.802967 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:32.803934 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:32.803944 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:32.803946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:32.804088 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:32.804521 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:32.804525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:32.804527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:32.804901 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:13:32.805135 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:32.805542 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 03:13:32.805714 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 03:13:32.806813 24233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:13:32.956965 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 24092
I20260811 03:13:32.977653 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 24245
Aug 11 03:13:32 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:13:32.994694 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:32.978523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.11.60.2:40479)
0811 03:13:32.978713 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:32.978719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:32.978741 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:13:32.978887 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:32.978893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:32.978984 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:32.979090 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:13:32.980436 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.981172 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.982566 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:32.983004 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:32.983143 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:32.988158 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:13:32.988176 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:13:32.988194 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:13:32.988235 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:13:32.991514 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:32.992303 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:32.992311 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:32.992313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:32.992489 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:32.992886 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:32.992891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:32.992894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:32.993289 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:13:32.993604 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:32.993977 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 03:13:32.994206 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 03:13:32.995589 24388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:13:33.089309 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 24245
I20260811 03:13:33.116032 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 24401
Aug 11 03:13:33 dist-test-slave-tm1g krb5kdc[23482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:13:33.134536 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:13:33.116850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.11.60.3:46141)
0811 03:13:33.117067 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 03:13:33.117074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:13:33.117097 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:13:33.117198 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:13:33.117204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:13:33.117301 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:13:33.117415 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:13:33.118853 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:33.120118 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:33.121392 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:13:33.122146 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:13:33.122323 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:13:33.127055 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:13:33.127069 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:13:33.127090 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:13:33.127146 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:13:33.131498 (+ 4352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:33.132175 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:33.132186 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:33.132189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:33.132374 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:13:33.132719 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:13:33.132724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:13:33.132729 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:13:33.133276 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:13:33.133630 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 03:13:33.133826 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 03:13:33.134104 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:13:33.135345 24534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:13:33.261858 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 24401
I20260811 03:13:33.284250 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 23966
I20260811 03:13:33.285517 24038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:13:33.400668 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 23966
I20260811 03:13:33.426573 11504 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 03:13:33.771744 11504 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 03:13:33.772003 11504 mini_ranger.cc:66] Stopping Ranger...
I20260811 03:13:34.117606 11504 mini_ranger.cc:68] Stopped Ranger
2026-08-11 03:13:34.117 UTC [23513] LOG: received smart shutdown request
2026-08-11 03:13:34.121 UTC [23513] LOG: background worker "logical replication launcher" (PID 23523) exited with exit code 1
2026-08-11 03:13:34.122 UTC [23518] LOG: shutting down
2026-08-11 03:13:34.297 UTC [23513] LOG: database system is shut down
2026-08-11T03:13:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52396 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:13:34 dist-test-slave-tm1g krb5kdc[24602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:13:34 dist-test-slave-tm1g krb5kdc[24602](info): set up 2 sockets
Aug 11 03:13:34 dist-test-slave-tm1g krb5kdc[24602](info): commencing operation
krb5kdc: starting...
W20260811 03:13:35.609083 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:13:35 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:13:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:13:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.60.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.60.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/rangeradmin_127.11.60.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/rangeradmin_127.11.60.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.60.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.60.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/rangerlookup_127.11.60.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/rangerlookup_127.11.60.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.60.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.60.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/HTTP_127.11.60.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/HTTP_127.11.60.20@KRBTEST.COM.keytab.
I20260811 03:13:36.013048 11504 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRu2AW0/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/postgres -l logfile start
2026-08-11 03:13:38.638 UTC [24635] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 03:13:38.638 UTC [24635] LOG: listening on IPv4 address "127.11.60.20", port 46441
2026-08-11 03:13:38.642 UTC [24635] LOG: listening on Unix socket "/tmp/.s.PGSQL.46441"
2026-08-11 03:13:38.669 UTC [24639] LOG: database system was shut down at 2026-08-11 03:13:37 UTC
2026-08-11 03:13:38.673 UTC [24635] LOG: database system is ready to accept connections
I20260811 03:13:39.717381 11504 mini_postgres.cc:96] Postgres bound to 46441
2026-08-11 03:13:39.742 UTC [24648] FATAL: database "slave" does not exist
127.11.60.20:46441 - accepting connections
I20260811 03:13:39.742859 11504 mini_ranger.cc:161] Starting Ranger...
I20260811 03:13:39.777122 11504 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 03:13:40.332444 11504 mini_ranger.cc:87] Created ranger Postgres database
I20260811 03:13:40.332580 11504 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-admin
2026-08-11 03:13:40,635 [I] DB FLAVOR :POSTGRES
2026-08-11 03:13:40,635 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 03:13:40,636 [I] Checking connection..
2026-08-11 03:13:40,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 03:13:40,912 [I] Checking connection passed.
2026-08-11 03:13:40,912 [I] --------- Verifying version history table ---------
2026-08-11 03:13:40,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 03:13:41,172 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 03:13:41,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 03:13:41,443 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 03:13:41,444 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 03:13:41,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 03:13:41.699 UTC [24717] WARNING: there is no transaction in progress
2026-08-11 03:13:41,708 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 03:13:41,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 03:13:41,962 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 03:13:41,963 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 03:13:41,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 03:13:42,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 03:13:42,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tm1g.c.gcp-upstream.internal','N') ;"
2026-08-11 03:13:42,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 03:13:42,989 [I] Table x_portal_user does not exist in database ranger
2026-08-11 03:13:42,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 03:13:43,248 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 03:13:43,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 03:13:43,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 03:13:43,769 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 03:13:43,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 03:13:44.777 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.794 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.811 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.828 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.847 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.874 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.930 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.937 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.952 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.963 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.975 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:44.990 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:45.000 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:45.006 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:45.013 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:45.020 UTC [24837] WARNING: there is no transaction in progress
2026-08-11 03:13:45,691 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 03:13:45,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 03:13:45,943 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 03:13:45,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 03:13:46,187 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 03:13:46,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 03:13:46,457 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 03:13:46,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 03:13:46,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 03:13:46,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tm1g.c.gcp-upstream.internal';"
2026-08-11 03:13:47,222 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 03:13:47,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 03:13:47,480 [I] DB_PATCHES have already been applied
I20260811 03:13:47.487067 11504 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRu2AW0/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 03:13:47.487167 11504 mini_ranger.cc:193] Using host: 127.11.60.20
I20260811 03:13:47.507851 11504 mini_ranger.cc:227] Ranger admin URL: http://127.11.60.20:59807
Aug 11, 2026 3:13:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 3:13:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 3:13:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 3:13:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 3:13:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 3:13:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 3:13:48 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 03:13:48.705914 11504 mini_ranger.cc:160] Time spent starting Ranger: real 8.963s user 0.002s sys 0.059s
Aug 11, 2026 3:14:09 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 3:14:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 3:14:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 3:14:09 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 3:14:10 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 03:14:11.043238 11504 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.60.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.60.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/rangerkms_127.11.60.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.60.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/rangerkms_127.11.60.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.60.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.60.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.60.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/HTTP_127.11.60.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.60.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/HTTP_127.11.60.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 03:14:11 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418051, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 03:14:11.247107 11504 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 03:14:11.271555 11504 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 03:14:12.198838 11504 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 03:14:12.198974 11504 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-kms
I20260811 03:14:12.199035 11504 mini_ranger_kms.cc:227] Using postgres at 127.11.60.20:46441
2026-08-11 03:14:12,575 [I] DB FLAVOR :POSTGRES
2026-08-11 03:14:12,576 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 03:14:12,576 [I] Checking connection
2026-08-11 03:14:12,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 03:14:12,844 [I] connection success
2026-08-11 03:14:12,845 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 03:14:12,845 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 03:14:12,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 03:14:13,090 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 03:14:13,091 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 03:14:13,091 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 03:14:13,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.60.20:46441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 03:14:13,361 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 03:14:13.810273 11504 mini_ranger_kms.cc:310] Created kms service
I20260811 03:14:13.984294 11504 mini_ranger_kms.cc:326] Created kudu user
I20260811 03:14:14.032829 11504 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 03:14:14 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418051, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:14:14.459069 11504 mini_ranger_kms.cc:384] Added ranger policy
I20260811 03:14:14.459372 11504 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRu2AW0/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRu2AW0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 03:14:14.459438 11504 mini_ranger_kms.cc:240] Using host: 127.11.60.20
I20260811 03:14:14.479561 11504 mini_ranger_kms.cc:276] Ranger KMS PID: 25036
I20260811 03:14:14.479696 11504 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.60.20:48007
Aug 11, 2026 3:14:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 3:14:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 3:14:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 3:14:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 3:14:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 3:14:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 03:14:16 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786418056, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.60.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:16.956439 11504 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.709s user 0.073s sys 0.336s
I20260811 03:14:16.956696 11504 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 03:14:16.956864 11504 mini_ranger_kms.cc:401] 127.11.60.20:48007/kms/v1/keys
Aug 11 03:14:17 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418056, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.60.20@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
Aug 11 03:14:17 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418056, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.60.20@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
Aug 11 03:14:19 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:14:19.998103 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:41997
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:38853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:41997
--ranger_config_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:48007/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:20.341542 25090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:20.341846 25090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:20.342003 25090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:20.349541 25090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:14:20.349663 25090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:20.349772 25090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:20.349871 25090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:14:20.349956 25090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:14:20.360518 25090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:38853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.60.20:48007/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:41997
--ranger_config_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:41997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:20.361896 25090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:20.363310 25090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:20.375818 25098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:20.375788 25099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:20.375869 25101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:20.376621 25090 server_base.cc:1034] running on GCE node
I20260811 03:14:20.377399 25090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:20.378932 25090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:20.380160 25090 hybrid_clock.cc:648] HybridClock initialized: now 1786418060380120 us; error 29 us; skew 500 ppm
I20260811 03:14:20.380520 25090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:20 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418060, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:20.389412 25090 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:14:20.391016 25090 webserver.cc:460] Webserver started at http://127.11.60.62:36059/ using document root <none> and password file <none>
I20260811 03:14:20.391724 25090 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:20.391871 25090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:20.392199 25090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 03:14:20 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418060, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:14:20.453503 25090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "b9dc20f6028544508298b5b5342fa2a3"
format_stamp: "Formatted at 2026-08-11 03:14:20 on dist-test-slave-tm1g"
server_key: "bed3fb3fab5effbc46f8be92a1a4c66d"
server_key_iv: "ce594764076984b38316d03ea150b56a"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:20.454792 25090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "b9dc20f6028544508298b5b5342fa2a3"
format_stamp: "Formatted at 2026-08-11 03:14:20 on dist-test-slave-tm1g"
server_key: "bed3fb3fab5effbc46f8be92a1a4c66d"
server_key_iv: "ce594764076984b38316d03ea150b56a"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:20.462167 25090 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260811 03:14:20.485514 25110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:20.487228 25090 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 03:14:20.487448 25090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "b9dc20f6028544508298b5b5342fa2a3"
format_stamp: "Formatted at 2026-08-11 03:14:20 on dist-test-slave-tm1g"
server_key: "bed3fb3fab5effbc46f8be92a1a4c66d"
server_key_iv: "ce594764076984b38316d03ea150b56a"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:20.487709 25090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:20.502002 25090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:20.506242 25090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:20.506635 25090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:20.526351 25090 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:41997
I20260811 03:14:20.526386 25166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:41997 every 8 connection(s)
I20260811 03:14:20.527840 25090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:14:20.531756 25167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:20.535981 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 25090
I20260811 03:14:20.536221 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:14:20.536902 11504 external_mini_cluster.cc:1428] Setting key 94f9d1158174d5966cd294b88b8eec47
I20260811 03:14:20.544140 25167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: Bootstrap starting.
I20260811 03:14:20.547983 25167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:20.549288 25167 log.cc:824] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:20.552363 25167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: No bootstrap required, opened a new log
Aug 11 03:14:20 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:20.562685 25170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:20.539001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.11.60.62:41997)
0811 03:14:20.539898 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:20.539915 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:20.539969 (+ 54us) server_negotiation.cc:407] Connection header received
0811 03:14:20.542536 (+ 2567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:20.542565 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:20.543066 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:20.543493 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:20.544786 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:20.546898 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:20.548401 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:20.549495 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:20.549675 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:20.553847 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:20.553858 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:20.553876 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:20.553934 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:20.557935 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:20.558785 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:20.558794 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:20.558803 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:20.559010 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:20.559380 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:20.559385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:20.559389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:20.560176 (+ 787us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:20.560500 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:20.560788 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 03:14:20.561242 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":367,"threads_started":1}
I20260811 03:14:20.563421 25167 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } }
I20260811 03:14:20.563771 25167 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:20.563896 25167 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9dc20f6028544508298b5b5342fa2a3, State: Initialized, Role: FOLLOWER
I20260811 03:14:20.564504 25167 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } }
I20260811 03:14:20.564765 25167 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:14:20.564919 25167 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:14:20.565115 25167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:20.567440 25167 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } }
I20260811 03:14:20.567936 25167 leader_election.cc:302] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9dc20f6028544508298b5b5342fa2a3; no voters:
I20260811 03:14:20.568532 25167 leader_election.cc:288] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:14:20.568665 25172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:14:20.570559 25172 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 1 LEADER]: Becoming Leader. State: Replica: b9dc20f6028544508298b5b5342fa2a3, State: Running, Role: LEADER
I20260811 03:14:20.571215 25167 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:14:20.571156 25172 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } }
I20260811 03:14:20.574420 25173 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9dc20f6028544508298b5b5342fa2a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } } }
I20260811 03:14:20.574383 25174 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9dc20f6028544508298b5b5342fa2a3. Latest consensus state: current_term: 1 leader_uuid: "b9dc20f6028544508298b5b5342fa2a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } } }
I20260811 03:14:20.574865 25173 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:20.574894 25174 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:20.580137 25180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:14:20.580752 25167 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 03:14:20.593850 25180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:14:20.609668 25180 catalog_manager.cc:1347] Generated new cluster ID: 1b07991ac8b049148163c779e1b584a0
I20260811 03:14:20.609885 25180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:14:20.617898 25180 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:14:20.619382 25180 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:14:20.625491 25180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: Generated new TSK 0
I20260811 03:14:20.626991 25180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:14:21.443825 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:48007/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41997
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:38853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:14:21 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786418061, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 03:14:21.867956 25216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:21.868438 25216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:21.868645 25216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:21.877892 25216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:21.878050 25216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:21.878328 25216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:14:21.890177 25216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:38853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.60.20:48007/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:21.892086 25216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:21.893594 25216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:21.911764 25226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:21.912015 25228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:21.918146 25216 server_base.cc:1034] running on GCE node
W20260811 03:14:21.919749 25231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:21.920589 25216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:21.937183 25216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:21.940666 25216 hybrid_clock.cc:648] HybridClock initialized: now 1786418061940568 us; error 82 us; skew 500 ppm
I20260811 03:14:21.941234 25216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:21 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418061, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:21.945212 25216 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:14:21.954566 25216 webserver.cc:460] Webserver started at http://127.11.60.1:43911/ using document root <none> and password file <none>
I20260811 03:14:21.955374 25216 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:21.955530 25216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:21.955935 25216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 03:14:21 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418061, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:14:21.976590 25216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "085e3567078b4fb9a556d1ca9fb1e0ab"
format_stamp: "Formatted at 2026-08-11 03:14:21 on dist-test-slave-tm1g"
server_key: "c23c866df0c3ffb0881d69d914a8583d"
server_key_iv: "4e433cecabbbe8f4baaae02ce7ec71c0"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:21.977825 25216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "085e3567078b4fb9a556d1ca9fb1e0ab"
format_stamp: "Formatted at 2026-08-11 03:14:21 on dist-test-slave-tm1g"
server_key: "c23c866df0c3ffb0881d69d914a8583d"
server_key_iv: "4e433cecabbbe8f4baaae02ce7ec71c0"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:21.985334 25216 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.003s
I20260811 03:14:22.026837 25239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:22.028746 25216 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.010s sys 0.005s
I20260811 03:14:22.028999 25216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "085e3567078b4fb9a556d1ca9fb1e0ab"
format_stamp: "Formatted at 2026-08-11 03:14:21 on dist-test-slave-tm1g"
server_key: "c23c866df0c3ffb0881d69d914a8583d"
server_key_iv: "4e433cecabbbe8f4baaae02ce7ec71c0"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:22.029400 25216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:22.059903 25216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:22.064627 25216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:22.064996 25216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:22.067355 25216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:22.079202 25216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:22.079387 25216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 03:14:22.079595 25216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:22.079696 25216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:22.139518 25216 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:46023
I20260811 03:14:22.139540 25370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:46023 every 8 connection(s)
I20260811 03:14:22.141247 25216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:14:22.144029 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 25216
I20260811 03:14:22.144304 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:14:22.145352 11504 external_mini_cluster.cc:1428] Setting key e816ac47dae9d59aa23743f33e827217
Aug 11 03:14:22 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418061, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
Aug 11 03:14:22 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418061, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
I20260811 03:14:22.252938 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:22.205229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:33865 (local address 127.11.60.62:41997)
0811 03:14:22.225485 (+ 20256us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:22.225492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:22.225515 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:14:22.225587 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:22.225593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:22.225706 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:22.225828 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:22.234445 (+ 8617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:22.235591 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:22.239399 (+ 3808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:22.240073 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:22.240223 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:22.244875 (+ 4652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:22.244882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:22.244887 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:22.244953 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:22.248664 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:22.249581 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:22.249587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:22.249591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:22.249752 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:22.250479 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:22.250485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:22.250489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:22.251107 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:22.251265 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:22.252236 (+ 971us) server_negotiation.cc:299] Negotiation successful
0811 03:14:22.252474 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20095,"thread_start_us":20731,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
I20260811 03:14:22.254249 25374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:22.205868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:41997 (local address 127.11.60.1:33865)
0811 03:14:22.220133 (+ 14265us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:22.220202 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:22.224117 (+ 3915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:22.226107 (+ 1990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:22.226122 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:22.226902 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:22.227833 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:22.227861 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:22.236571 (+ 8710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:22.236578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:22.239228 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:22.239252 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:22.240355 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:22.240364 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:22.240673 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:22.241543 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:22.241574 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:22.244590 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:22.248825 (+ 4235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:22.248834 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:22.248855 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:22.249452 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:22.250007 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:22.250013 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:22.250016 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:22.250337 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:22.251260 (+ 923us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:22.251270 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:22.251744 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 03:14:22.252103 (+ 359us) client_negotiation.cc:240] Negotiation successful
0811 03:14:22.252779 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13851,"thread_start_us":16328,"threads_started":1}
I20260811 03:14:22.269137 25371 heartbeater.cc:347] Connected to a master server at 127.11.60.62:41997
I20260811 03:14:22.269544 25371 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:22.270516 25371 heartbeater.cc:511] Master 127.11.60.62:41997 requested a full tablet report, sending...
I20260811 03:14:22.273324 25129 ts_manager.cc:194] Registered new tserver with Master: 085e3567078b4fb9a556d1ca9fb1e0ab (127.11.60.1:46023)
I20260811 03:14:22.275461 25129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:33865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
W20260811 03:14:22.311323 11504 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.60.2: real 0.151s user 0.000s sys 0.028s
I20260811 03:14:22.312124 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:48007/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41997
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:38853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:14:22 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418061, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
W20260811 03:14:22.826813 25377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:22.827380 25377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:22.827577 25377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:22.843778 25377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:22.844028 25377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:22.844273 25377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:14:22.862056 25377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:38853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.60.20:48007/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:22.864166 25377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:22.866035 25377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:22.885589 25394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:22.890177 25393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:22.890561 25377 server_base.cc:1034] running on GCE node
W20260811 03:14:22.891265 25398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:22.892235 25377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:22.894177 25377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:22.895489 25377 hybrid_clock.cc:648] HybridClock initialized: now 1786418062895395 us; error 86 us; skew 500 ppm
I20260811 03:14:22.895852 25377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:22 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418062, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:22.899295 25377 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:14:22.901178 25377 webserver.cc:460] Webserver started at http://127.11.60.2:34591/ using document root <none> and password file <none>
I20260811 03:14:22.901933 25377 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:22.902083 25377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:22.902504 25377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 03:14:22 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418062, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:14:22.921541 25377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "02bd1146223e481595f632648d51f044"
format_stamp: "Formatted at 2026-08-11 03:14:22 on dist-test-slave-tm1g"
server_key: "6abfd6f44342032e7b530014d50c3316"
server_key_iv: "51a5f041d7484c99b2e8fa9849bf28e1"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:22.922748 25377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "02bd1146223e481595f632648d51f044"
format_stamp: "Formatted at 2026-08-11 03:14:22 on dist-test-slave-tm1g"
server_key: "6abfd6f44342032e7b530014d50c3316"
server_key_iv: "51a5f041d7484c99b2e8fa9849bf28e1"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:22.930444 25377 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260811 03:14:22.949075 25407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:22.951334 25377 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 03:14:22.951663 25377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "02bd1146223e481595f632648d51f044"
format_stamp: "Formatted at 2026-08-11 03:14:22 on dist-test-slave-tm1g"
server_key: "6abfd6f44342032e7b530014d50c3316"
server_key_iv: "51a5f041d7484c99b2e8fa9849bf28e1"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:22.952077 25377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:22.971249 25377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:22.976643 25377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:22.976984 25377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:22.978144 25377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:22.979959 25377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:22.980049 25377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:22.980154 25377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:22.980219 25377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:23.021107 25377 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:38989
I20260811 03:14:23.021126 25535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:38989 every 8 connection(s)
I20260811 03:14:23.022809 25377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:14:23.031888 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 25377
I20260811 03:14:23.032106 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:14:23.032740 11504 external_mini_cluster.cc:1428] Setting key 4095fcde6968290451792a3eff26193c
Aug 11 03:14:23 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418062, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:23.044715 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:23.025707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:60541 (local address 127.11.60.62:41997)
0811 03:14:23.026212 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:23.026217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:23.027239 (+ 1022us) server_negotiation.cc:407] Connection header received
0811 03:14:23.029126 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:23.029133 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:23.029250 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:23.029384 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:23.030963 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.032000 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.034288 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.034841 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.034981 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:23.038634 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:23.038639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:23.038642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:23.038674 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:23.041478 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:23.042227 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:23.042237 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:23.042239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:23.042430 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:23.042872 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:23.042877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:23.042880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:23.043189 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:23.043376 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:23.044092 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 03:14:23.044337 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":273,"threads_started":1}
I20260811 03:14:23.045868 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:23.026204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:41997 (local address 127.11.60.2:60541)
0811 03:14:23.027051 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:23.027096 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:23.028806 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:23.029627 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:23.029636 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:23.030073 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:23.030780 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:23.030798 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.032234 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.032239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:23.034141 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:23.034155 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.035081 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.035088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:23.035483 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:23.036247 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:23.036269 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:23.038404 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:23.041624 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:23.041633 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:23.041650 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:23.042094 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:23.042550 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:23.042555 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:23.042559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:23.042748 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:23.043331 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:23.043341 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:23.043699 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 03:14:23.043977 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 03:14:23.044520 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":373,"threads_started":1}
I20260811 03:14:23.047534 25536 heartbeater.cc:347] Connected to a master server at 127.11.60.62:41997
I20260811 03:14:23.047834 25536 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:23.048834 25536 heartbeater.cc:511] Master 127.11.60.62:41997 requested a full tablet report, sending...
I20260811 03:14:23.050541 25129 ts_manager.cc:194] Registered new tserver with Master: 02bd1146223e481595f632648d51f044 (127.11.60.2:38989)
I20260811 03:14:23.051426 25129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:60541
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:14:23.107048 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:48007/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41997
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:38853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:14:23.278987 25371 heartbeater.cc:503] Master 127.11.60.62:41997 was elected leader, sending a full tablet report...
W20260811 03:14:23.468183 25543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:23.468470 25543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:23.468677 25543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:23.476470 25543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:23.476619 25543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:23.476810 25543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:14:23.487023 25543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:38853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.60.20:48007/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:23.488494 25543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:23.490010 25543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:23.503970 25553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:23.504040 25551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:23.504271 25550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:23.505196 25543 server_base.cc:1034] running on GCE node
I20260811 03:14:23.505865 25543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:23.506927 25543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:23.508163 25543 hybrid_clock.cc:648] HybridClock initialized: now 1786418063508115 us; error 74 us; skew 500 ppm
I20260811 03:14:23.508486 25543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:23 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:23.511734 25543 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:14:23.513165 25543 webserver.cc:460] Webserver started at http://127.11.60.3:42863/ using document root <none> and password file <none>
I20260811 03:14:23.513875 25543 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:23.513983 25543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:23.514262 25543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 03:14:23 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:14:23.531716 25543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "52a4cabb390f4fc9b4b8995292861f68"
format_stamp: "Formatted at 2026-08-11 03:14:23 on dist-test-slave-tm1g"
server_key: "1ec6ec2a74ce5a57ccff3d04dd91b5e2"
server_key_iv: "2e536286d977d882c346f4d6c1e13467"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:23.532846 25543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "52a4cabb390f4fc9b4b8995292861f68"
format_stamp: "Formatted at 2026-08-11 03:14:23 on dist-test-slave-tm1g"
server_key: "1ec6ec2a74ce5a57ccff3d04dd91b5e2"
server_key_iv: "2e536286d977d882c346f4d6c1e13467"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:23.539855 25543 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260811 03:14:23.556982 25560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:23.558801 25543 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 03:14:23.558981 25543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "52a4cabb390f4fc9b4b8995292861f68"
format_stamp: "Formatted at 2026-08-11 03:14:23 on dist-test-slave-tm1g"
server_key: "1ec6ec2a74ce5a57ccff3d04dd91b5e2"
server_key_iv: "2e536286d977d882c346f4d6c1e13467"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:23.559231 25543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:23.579435 25543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:23.583850 25543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:23.584185 25543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:23.585335 25543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:23.586916 25543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:23.586995 25543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:23.587168 25543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:23.587220 25543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:23.622754 25543 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:34643
I20260811 03:14:23.622797 25689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:34643 every 8 connection(s)
I20260811 03:14:23.624111 25543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:14:23.628451 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 25543
I20260811 03:14:23.628639 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:14:23.629108 11504 external_mini_cluster.cc:1428] Setting key 34ecc6005ee4707de6d5172ef7bb9fc8
Aug 11 03:14:23 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:23.644802 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:23.626757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:45209 (local address 127.11.60.62:41997)
0811 03:14:23.627292 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:23.627296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:23.628012 (+ 716us) server_negotiation.cc:407] Connection header received
0811 03:14:23.630134 (+ 2122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:23.630141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:23.630243 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:23.630405 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:23.632065 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.632955 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.634865 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.635333 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.635433 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:23.638657 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:23.638662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:23.638665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:23.638696 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:23.641553 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:23.642206 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:23.642209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:23.642211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:23.642395 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:23.642792 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:23.642796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:23.642799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:23.643203 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:23.643354 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:23.644191 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 03:14:23.644389 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":305,"threads_started":1}
I20260811 03:14:23.645824 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:23.627132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:41997 (local address 127.11.60.3:45209)
0811 03:14:23.627846 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:23.627885 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:23.629816 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:23.630658 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:23.630666 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:23.631074 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:23.631870 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:23.631898 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.633110 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.633114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:23.634718 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:23.634730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.635520 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.635525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:23.635740 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:23.636370 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:23.636392 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:23.638459 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:23.641678 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:23.641684 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:23.641696 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:23.642106 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:23.642489 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:23.642492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:23.642494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:23.642677 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:23.643352 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:23.643360 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:23.643771 (+ 411us) client_negotiation.cc:769] Sending connection context
0811 03:14:23.644094 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 03:14:23.644612 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":293,"threads_started":1}
I20260811 03:14:23.647248 25690 heartbeater.cc:347] Connected to a master server at 127.11.60.62:41997
I20260811 03:14:23.647569 25690 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:23.648291 25690 heartbeater.cc:511] Master 127.11.60.62:41997 requested a full tablet report, sending...
I20260811 03:14:23.649780 25129 ts_manager.cc:194] Registered new tserver with Master: 52a4cabb390f4fc9b4b8995292861f68 (127.11.60.3:34643)
I20260811 03:14:23.650664 25129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:45209
I20260811 03:14:23.657027 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:14:23.658857 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 25216
Aug 11 03:14:23 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:14:23.675634 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:23.659929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54768 (local address 127.11.60.1:46023)
0811 03:14:23.660609 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:23.660616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:23.660642 (+ 26us) server_negotiation.cc:407] Connection header received
0811 03:14:23.660700 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:23.660707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:23.660912 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:23.661041 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:23.662655 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.663848 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.665299 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.665842 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.665946 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:23.670234 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:23.670246 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:23.670261 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:23.670294 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:23.673186 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:23.673853 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:23.673860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:23.673862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:23.674000 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:23.674342 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:23.674349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:23.674352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:23.674684 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:23.674922 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:23.675146 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 03:14:23.675351 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":352,"threads_started":1}
I20260811 03:14:23.677101 25366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:23.766135 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 25216
I20260811 03:14:23.786029 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 25377
Aug 11 03:14:23 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:14:23.807881 25697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:23.787144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35282 (local address 127.11.60.2:38989)
0811 03:14:23.787696 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:23.787703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:23.787729 (+ 26us) server_negotiation.cc:407] Connection header received
0811 03:14:23.787798 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:23.787805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:23.788015 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:23.788162 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:23.789327 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.790365 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.791823 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.792547 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.792661 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:23.801625 (+ 8964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:23.801638 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:23.801654 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:23.801693 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:23.804969 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:23.805754 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:23.805762 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:23.805764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:23.805903 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:23.806232 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:23.806235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:23.806237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:23.806650 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:23.806880 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:23.807177 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 03:14:23.807496 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":294,"threads_started":1}
I20260811 03:14:23.809178 25531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:23.893168 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 25377
I20260811 03:14:23.913116 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 25543
Aug 11 03:14:23 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:14:23.930127 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:23.914353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35118 (local address 127.11.60.3:34643)
0811 03:14:23.915022 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:23.915030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:23.915059 (+ 29us) server_negotiation.cc:407] Connection header received
0811 03:14:23.915120 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:23.915126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:23.915366 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:23.915504 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:23.916741 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.917807 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.919277 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:23.919856 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:23.919964 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:23.924544 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:23.924570 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:23.924584 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:23.924618 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:23.927568 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:23.928211 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:23.928218 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:23.928220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:23.928345 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:23.928674 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:23.928677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:23.928679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:23.929042 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:23.929294 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:23.929528 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 03:14:23.929737 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":355,"threads_started":1}
I20260811 03:14:23.931484 25684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:24.016983 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 25543
I20260811 03:14:24.035625 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 25090
I20260811 03:14:24.037007 25162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:24.145933 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 25090
I20260811 03:14:24.173112 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:41997
--webserver_interface=127.11.60.62
--webserver_port=36059
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:38853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:41997
--ranger_config_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:48007/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.60.20:48007/kms with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:24.509662 25703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:24.510002 25703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:24.510172 25703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:24.517493 25703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:14:24.517647 25703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:24.517753 25703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:24.517854 25703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:14:24.517946 25703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:14:24.528275 25703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:38853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.60.20:48007/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:41997
--ranger_config_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:41997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=36059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:24.529644 25703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:24.531049 25703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:24.542987 25710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:24.543079 25709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:24.543074 25713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:24.543923 25703 server_base.cc:1034] running on GCE node
I20260811 03:14:24.544601 25703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:24.545945 25703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:24.547140 25703 hybrid_clock.cc:648] HybridClock initialized: now 1786418064547089 us; error 53 us; skew 500 ppm
I20260811 03:14:24.547500 25703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:24 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418064, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:24.550654 25703 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:14:24.552168 25703 webserver.cc:460] Webserver started at http://127.11.60.62:36059/ using document root <none> and password file <none>
I20260811 03:14:24.552837 25703 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:24.552989 25703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 03:14:24 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418064, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:14:24.603677 25703 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 03:14:24.606894 25723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:24.608330 25703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:14:24.608528 25703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "b9dc20f6028544508298b5b5342fa2a3"
format_stamp: "Formatted at 2026-08-11 03:14:20 on dist-test-slave-tm1g"
server_key: "bed3fb3fab5effbc46f8be92a1a4c66d"
server_key_iv: "ce594764076984b38316d03ea150b56a"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:24.609598 25703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:24.655295 25703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:24.659272 25703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:24.659606 25703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:24.678570 25703 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:41997
I20260811 03:14:24.678592 25782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:41997 every 8 connection(s)
I20260811 03:14:24.680029 25703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:14:24.686779 25783 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:14:24.690024 25783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: Bootstrap starting.
I20260811 03:14:24.690548 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 25703
I20260811 03:14:24.691679 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:48007/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:46023
--local_ip_for_outbound_sockets=127.11.60.1
--tserver_master_addrs=127.11.60.62:41997
--webserver_port=43911
--webserver_interface=127.11.60.1
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:38853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.60.20:48007/kms with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:14:24.712165 25783 log.cc:824] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:24.721472 25783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:14:24.722028 25783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: Bootstrap complete.
I20260811 03:14:24.731120 25783 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } }
I20260811 03:14:24.731935 25783 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9dc20f6028544508298b5b5342fa2a3, State: Initialized, Role: FOLLOWER
I20260811 03:14:24.732620 25783 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } }
I20260811 03:14:24.732856 25783 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:14:24.732985 25783 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:14:24.733184 25783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 1 FOLLOWER]: Advancing to term 2
I20260811 03:14:24.737206 25783 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } }
I20260811 03:14:24.737668 25783 leader_election.cc:302] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9dc20f6028544508298b5b5342fa2a3; no voters:
I20260811 03:14:24.738377 25783 leader_election.cc:288] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 03:14:24.738502 25788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 03:14:24.739786 25788 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [term 2 LEADER]: Becoming Leader. State: Replica: b9dc20f6028544508298b5b5342fa2a3, State: Running, Role: LEADER
I20260811 03:14:24.740384 25788 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } }
I20260811 03:14:24.740644 25783 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:14:24.743098 25789 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b9dc20f6028544508298b5b5342fa2a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } } }
I20260811 03:14:24.743618 25789 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:24.747812 25790 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9dc20f6028544508298b5b5342fa2a3. Latest consensus state: current_term: 2 leader_uuid: "b9dc20f6028544508298b5b5342fa2a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dc20f6028544508298b5b5342fa2a3" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41997 } } }
I20260811 03:14:24.748185 25790 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:24.749008 25794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:14:24.755892 25794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:14:24.757310 25794 catalog_manager.cc:1259] Loaded cluster ID: 1b07991ac8b049148163c779e1b584a0
I20260811 03:14:24.757463 25794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:14:24.761883 25794 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:14:24.763550 25783 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 03:14:24.763696 25794 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: Loaded TSK: 0
I20260811 03:14:24.765285 25794 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 03:14:25.139500 25786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:25.139976 25786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:25.140184 25786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:25.151917 25786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:25.152117 25786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:25.152359 25786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:14:25.166109 25786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:38853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.60.20:48007/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:46023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=43911
--tserver_master_addrs=127.11.60.62:41997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:25.167622 25786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:25.169380 25786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:25.214934 25825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:25.217505 25824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:25.218628 25827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:25.224045 25786 server_base.cc:1034] running on GCE node
I20260811 03:14:25.225112 25786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:25.226470 25786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:25.227762 25786 hybrid_clock.cc:648] HybridClock initialized: now 1786418065227703 us; error 55 us; skew 500 ppm
I20260811 03:14:25.228111 25786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:25 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418065, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:25.231904 25786 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:14:25.241279 25786 webserver.cc:460] Webserver started at http://127.11.60.1:43911/ using document root <none> and password file <none>
I20260811 03:14:25.242355 25786 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:25.242527 25786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 03:14:25 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418065, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:14:25.272080 25786 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.002s sys 0.005s
I20260811 03:14:25.276135 25836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:25.278141 25786 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 03:14:25.278388 25786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "085e3567078b4fb9a556d1ca9fb1e0ab"
format_stamp: "Formatted at 2026-08-11 03:14:21 on dist-test-slave-tm1g"
server_key: "c23c866df0c3ffb0881d69d914a8583d"
server_key_iv: "4e433cecabbbe8f4baaae02ce7ec71c0"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:25.279446 25786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:25.303218 25786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:25.308465 25786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:25.308825 25786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:25.310045 25786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:25.311715 25786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:25.311841 25786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:25.311970 25786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:25.312072 25786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:25.380162 25786 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:46023
I20260811 03:14:25.380218 25961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:46023 every 8 connection(s)
I20260811 03:14:25.382261 25786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:14:25.392192 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 25786
I20260811 03:14:25.393052 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:48007/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:38989
--local_ip_for_outbound_sockets=127.11.60.2
--tserver_master_addrs=127.11.60.62:41997
--webserver_port=34591
--webserver_interface=127.11.60.2
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:38853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.60.20:48007/kms with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:14:25 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418065, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:25.432823 25964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:25.387686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:41997 (local address 127.11.60.1:42651)
0811 03:14:25.407016 (+ 19330us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:25.407080 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:25.408865 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:25.410453 (+ 1588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:25.410462 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:25.411082 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:25.411648 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:25.411666 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:25.414191 (+ 2525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:25.414197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:25.416241 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:25.416256 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:25.417868 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:25.417875 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:25.418140 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:25.418921 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:25.418949 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:25.421752 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:25.426901 (+ 5149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:25.426913 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:25.426933 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:25.427641 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:25.428279 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:25.428285 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:25.428291 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:25.428546 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:25.429812 (+ 1266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:25.429823 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:25.430283 (+ 460us) client_negotiation.cc:769] Sending connection context
0811 03:14:25.430665 (+ 382us) client_negotiation.cc:240] Negotiation successful
0811 03:14:25.431311 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19000,"thread_start_us":18740,"threads_started":1}
I20260811 03:14:25.435232 25965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:25.394917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:42651 (local address 127.11.60.62:41997)
0811 03:14:25.406282 (+ 11365us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:25.406336 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:25.409182 (+ 2846us) server_negotiation.cc:407] Connection header received
0811 03:14:25.409523 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:25.409535 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:25.409891 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:25.410188 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:25.411873 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:25.413965 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:25.416402 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:25.417616 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:25.417815 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:25.422061 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:25.422073 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:25.422096 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:25.422149 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:25.426701 (+ 4552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:25.427838 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:25.427847 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:25.427858 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:25.428107 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:25.428705 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:25.428713 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:25.428719 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:25.429482 (+ 763us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:25.429721 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:25.433668 (+ 3947us) server_negotiation.cc:299] Negotiation successful
0811 03:14:25.434073 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11060,"thread_start_us":10820,"threads_started":1}
I20260811 03:14:25.438207 25962 heartbeater.cc:347] Connected to a master server at 127.11.60.62:41997
I20260811 03:14:25.438560 25962 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:25.439396 25962 heartbeater.cc:511] Master 127.11.60.62:41997 requested a full tablet report, sending...
W20260811 03:14:25.442690 25962 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:41997 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 03:14:25.442917 25962 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:25.443197 25962 heartbeater.cc:511] Master 127.11.60.62:41997 requested a full tablet report, sending...
I20260811 03:14:25.444480 25962 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:25.444736 25962 heartbeater.cc:511] Master 127.11.60.62:41997 requested a full tablet report, sending...
W20260811 03:14:25.445806 25962 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 03:14:25.824998 25966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:25.825474 25966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:25.825671 25966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:25.833976 25966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:25.834120 25966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:25.834350 25966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:14:25.849785 25966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:38853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.60.20:48007/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:38989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=34591
--tserver_master_addrs=127.11.60.62:41997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:25.851615 25966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:25.853305 25966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:25.869334 25982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:25.869963 25984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:25.870074 25981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:25.871621 25966 server_base.cc:1034] running on GCE node
I20260811 03:14:25.872344 25966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:25.873400 25966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:25.874709 25966 hybrid_clock.cc:648] HybridClock initialized: now 1786418065874641 us; error 57 us; skew 500 ppm
I20260811 03:14:25.875051 25966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:25 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418065, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:25.878257 25966 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:14:25.880079 25966 webserver.cc:460] Webserver started at http://127.11.60.2:34591/ using document root <none> and password file <none>
I20260811 03:14:25.880883 25966 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:25.881052 25966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 03:14:25 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418065, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:14:25.906859 25966 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.003s sys 0.004s
I20260811 03:14:25.927003 25992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:25.929054 25966 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.004s sys 0.016s
I20260811 03:14:25.929339 25966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "02bd1146223e481595f632648d51f044"
format_stamp: "Formatted at 2026-08-11 03:14:22 on dist-test-slave-tm1g"
server_key: "6abfd6f44342032e7b530014d50c3316"
server_key_iv: "51a5f041d7484c99b2e8fa9849bf28e1"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:25.931022 25966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:25.953521 25966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:25.959312 25966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:25.959688 25966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:25.970868 25966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:25.972536 25966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:25.972675 25966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:25.972812 25966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:25.972950 25966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:26.018777 25966 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:38989
I20260811 03:14:26.018828 26122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:38989 every 8 connection(s)
I20260811 03:14:26.020556 25966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:14:26.023401 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 25966
I20260811 03:14:26.024478 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.60.20:48007/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:34643
--local_ip_for_outbound_sockets=127.11.60.3
--tserver_master_addrs=127.11.60.62:41997
--webserver_port=42863
--webserver_interface=127.11.60.3
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:38853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.60.20:48007/kms with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:14:26 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786418066, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 03:14:26 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418065, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:26.081596 26126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.033156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:41997 (local address 127.11.60.2:34873)
0811 03:14:26.045696 (+ 12540us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:26.045759 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:26.048619 (+ 2860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:26.061131 (+ 12512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:26.061146 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:26.061830 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:26.062849 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:26.062876 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.064685 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.064691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:26.066798 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:26.066813 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.068202 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.068209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:26.068532 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:26.069402 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:26.069445 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:26.072299 (+ 2854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:26.076488 (+ 4189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:26.076498 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:26.076518 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:26.077166 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:26.077663 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:26.077669 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:26.077675 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:26.077913 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:26.078689 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:26.078699 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:26.079135 (+ 436us) client_negotiation.cc:769] Sending connection context
0811 03:14:26.079479 (+ 344us) client_negotiation.cc:240] Negotiation successful
0811 03:14:26.080070 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12188,"thread_start_us":15752,"threads_started":1}
I20260811 03:14:26.082757 26127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.033841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:34873 (local address 127.11.60.62:41997)
0811 03:14:26.060218 (+ 26377us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:26.060225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:26.060255 (+ 30us) server_negotiation.cc:407] Connection header received
0811 03:14:26.060331 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:26.060337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:26.060470 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:26.063062 (+ 2592us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:26.063247 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.064476 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.067032 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.067738 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.067866 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:26.072584 (+ 4718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:26.072591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:26.072596 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:26.072638 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:26.076286 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:26.077338 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:26.077344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:26.077347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:26.077514 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:26.078052 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:26.078057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:26.078060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:26.078457 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:26.078602 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:26.082110 (+ 3508us) server_negotiation.cc:299] Negotiation successful
0811 03:14:26.082381 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26224,"thread_start_us":26781,"threads_started":1}
I20260811 03:14:26.085338 26123 heartbeater.cc:347] Connected to a master server at 127.11.60.62:41997
I20260811 03:14:26.085695 26123 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:26.086665 26123 heartbeater.cc:511] Master 127.11.60.62:41997 requested a full tablet report, sending...
I20260811 03:14:26.088872 25743 ts_manager.cc:194] Registered new tserver with Master: 02bd1146223e481595f632648d51f044 (127.11.60.2:38989)
I20260811 03:14:26.091073 25743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:34873
Aug 11 03:14:26 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418066, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:14:26.445977 25962 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:26.446470 25962 heartbeater.cc:511] Master 127.11.60.62:41997 requested a full tablet report, sending...
I20260811 03:14:26.447801 25743 ts_manager.cc:194] Registered new tserver with Master: 085e3567078b4fb9a556d1ca9fb1e0ab (127.11.60.1:46023)
I20260811 03:14:26.449029 25743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:42651
W20260811 03:14:26.511801 26128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:26.512266 26128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:26.512440 26128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:26.522622 26128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:26.522831 26128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:26.523041 26128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:14:26.537746 26128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:38853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.60.20:48007/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:34643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=42863
--tserver_master_addrs=127.11.60.62:41997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:26.539778 26128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:26.541500 26128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:26.569161 26134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:26.569219 26135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:26.577625 26128 server_base.cc:1034] running on GCE node
W20260811 03:14:26.578292 26137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:26.579252 26128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:26.589797 26128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:26.591125 26128 hybrid_clock.cc:648] HybridClock initialized: now 1786418066591052 us; error 73 us; skew 500 ppm
I20260811 03:14:26.591593 26128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:26 dist-test-slave-tm1g krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418066, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:26.595884 26128 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:14:26.599231 26128 webserver.cc:460] Webserver started at http://127.11.60.3:42863/ using document root <none> and password file <none>
I20260811 03:14:26.600176 26128 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:26.600332 26128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 03:14:26 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418066, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
I20260811 03:14:26.644629 26128 fs_manager.cc:709] Time spent opening directory manager: real 0.017s user 0.012s sys 0.004s
I20260811 03:14:26.648428 26147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:26.650245 26128 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 03:14:26.650463 26128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "52a4cabb390f4fc9b4b8995292861f68"
format_stamp: "Formatted at 2026-08-11 03:14:23 on dist-test-slave-tm1g"
server_key: "1ec6ec2a74ce5a57ccff3d04dd91b5e2"
server_key_iv: "2e536286d977d882c346f4d6c1e13467"
server_key_version: "kuduclusterkey@0"
I20260811 03:14:26.651958 26128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:26.685868 26128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:26.691217 26128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:26.691622 26128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:26.692960 26128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:26.694794 26128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:26.694952 26128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 03:14:26.695101 26128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:26.695214 26128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:26.753810 26128 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:34643
I20260811 03:14:26.753899 26275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:34643 every 8 connection(s)
I20260811 03:14:26.755913 26128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:14:26.765197 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 26128
Aug 11 03:14:26 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418066, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for HTTP/127.11.60.20@KRBTEST.COM
Aug 11 03:14:26 dist-test-slave-tm1g krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418066, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:26.832859 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.766243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.11.60.62:41997)
0811 03:14:26.799866 (+ 33623us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:26.799873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:26.803175 (+ 3302us) server_negotiation.cc:407] Connection header received
0811 03:14:26.803249 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:26.803254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:26.803359 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:26.803481 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:26.816671 (+ 13190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.817830 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.819895 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.820552 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.820649 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:26.824715 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:26.824722 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:26.824726 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:26.824771 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:26.828875 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:26.829818 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:26.829827 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:26.829833 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:26.829988 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:26.830524 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:26.830530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:26.830534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:26.830941 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:26.831216 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:26.831823 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 03:14:26.832101 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33484,"thread_start_us":31599,"threads_started":1}
I20260811 03:14:26.839226 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.761439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:40763 (local address 127.11.60.62:41997)
0811 03:14:26.797821 (+ 36382us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:26.797829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:26.802364 (+ 4535us) server_negotiation.cc:407] Connection header received
0811 03:14:26.818479 (+ 16115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:26.818485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:26.818592 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:26.818674 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:26.822051 (+ 3377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.823327 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.826095 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.826965 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.827067 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:26.831787 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:26.831794 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:26.831799 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:26.831828 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:26.835286 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:26.836296 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:26.836305 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:26.836309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:26.836469 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:26.837045 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:26.837050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:26.837053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:26.837421 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:26.837565 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:26.838604 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0811 03:14:26.838827 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36212,"thread_start_us":38439,"threads_started":1}
I20260811 03:14:26.842039 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.760821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:41997 (local address 127.11.60.3:40763)
0811 03:14:26.800636 (+ 39815us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:26.800711 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:26.815119 (+ 14408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:26.819366 (+ 4247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:26.819382 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:26.820156 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:26.821099 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:26.821129 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.823634 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.823642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:26.825913 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:26.825932 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.827527 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.827536 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:26.827836 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:26.828609 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:26.828647 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:26.831474 (+ 2827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:26.835426 (+ 3952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:26.835435 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:26.835454 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:26.835945 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:26.836632 (+ 687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:26.836635 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:26.836638 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:26.836894 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:26.837681 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:26.837689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:26.838109 (+ 420us) client_negotiation.cc:769] Sending connection context
0811 03:14:26.839765 (+ 1656us) client_negotiation.cc:240] Negotiation successful
0811 03:14:26.840491 (+ 726us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39415,"thread_start_us":52123,"threads_started":1}
I20260811 03:14:26.843971 26276 heartbeater.cc:347] Connected to a master server at 127.11.60.62:41997
I20260811 03:14:26.844305 26276 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:26.845216 26276 heartbeater.cc:511] Master 127.11.60.62:41997 requested a full tablet report, sending...
I20260811 03:14:26.847079 25743 ts_manager.cc:194] Registered new tserver with Master: 52a4cabb390f4fc9b4b8995292861f68 (127.11.60.3:34643)
I20260811 03:14:26.848167 25743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:40763
I20260811 03:14:26.852615 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:14:26.919672 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.891344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57960 (local address 127.11.60.62:41997)
0811 03:14:26.891550 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:26.891556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:26.898937 (+ 7381us) server_negotiation.cc:407] Connection header received
0811 03:14:26.899452 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:26.899459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:26.899569 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:26.899645 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:26.901797 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.902906 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.908062 (+ 5156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:26.908668 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:26.908763 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:26.912463 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:26.912470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:26.912474 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:26.912517 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:26.915991 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:26.917123 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:26.917129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:26.917133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:26.917280 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:26.917740 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:26.917746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:26.917749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:26.918124 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:26.918394 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:26.919057 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 03:14:26.919299 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 03:14:26.926826 25743 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:14:26.938241 25743 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:14:27.011605 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.965016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35132 (local address 127.11.60.3:34643)
0811 03:14:26.974882 (+ 9866us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:26.974896 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:26.991102 (+ 16206us) server_negotiation.cc:407] Connection header received
0811 03:14:26.991338 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:26.991351 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:26.991711 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:26.991965 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:26.999415 (+ 7450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.001198 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.007416 (+ 6218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.009041 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.009160 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.009409 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:27.011035 (+ 1626us) server_negotiation.cc:299] Negotiation successful
0811 03:14:27.011222 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9642,"thread_start_us":9479,"threads_started":1}
I20260811 03:14:27.012735 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.964365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:34643 (local address 127.0.0.1:35132)
0811 03:14:26.990929 (+ 26564us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:26.990955 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:26.991155 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:26.998479 (+ 7324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:26.998492 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:26.998544 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:26.999122 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:26.999138 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.002143 (+ 3005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.002150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:27.005335 (+ 3185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:27.005357 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.009624 (+ 4267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.009630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:27.009855 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.009908 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 03:14:27.012120 (+ 2212us) client_negotiation.cc:240] Negotiation successful
0811 03:14:27.012344 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26401,"thread_start_us":26906,"threads_started":1}
I20260811 03:14:27.015580 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.957149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46023 (local address 127.0.0.1:54784)
0811 03:14:26.962931 (+ 5782us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:26.962978 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:26.963169 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:27.001386 (+ 38217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:27.001394 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:27.001431 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:27.001896 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:27.001913 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.010114 (+ 8201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.010118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:27.012384 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:27.012397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.014812 (+ 2415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.014817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:27.014986 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.015018 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 03:14:27.015087 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 03:14:27.015213 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5504,"mutex_wait_us":45,"thread_start_us":5349,"threads_started":1}
I20260811 03:14:27.016994 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.958168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54784 (local address 127.11.60.1:46023)
0811 03:14:26.999537 (+ 41369us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:26.999548 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:26.999585 (+ 37us) server_negotiation.cc:407] Connection header received
0811 03:14:26.999689 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:26.999699 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:26.999998 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:27.000218 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:27.005587 (+ 5369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.007258 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.013048 (+ 5790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.014567 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.016169 (+ 1602us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.016417 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:27.016490 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 03:14:27.016651 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41145,"thread_start_us":42233,"threads_started":1}
I20260811 03:14:27.019454 26200 tablet_service.cc:1467] Processing CreateTablet for tablet 55e799f39fe144b19482001d10a3f554 (DEFAULT_TABLE table=test-table [id=c14424e2da3f4f8f82fe669399bb7456]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:14:27.020720 25890 tablet_service.cc:1467] Processing CreateTablet for tablet 55e799f39fe144b19482001d10a3f554 (DEFAULT_TABLE table=test-table [id=c14424e2da3f4f8f82fe669399bb7456]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:14:27.022243 25890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e799f39fe144b19482001d10a3f554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:27.022964 26200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e799f39fe144b19482001d10a3f554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:27.038802 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.991785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:38989 (local address 127.0.0.1:35284)
0811 03:14:27.006977 (+ 15192us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:27.007019 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:27.007139 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:27.008110 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:27.008115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:27.008141 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:27.008538 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:27.008549 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.010400 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.010405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:27.033882 (+ 23477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:27.033904 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.036376 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.036385 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:27.036589 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.036624 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 03:14:27.038020 (+ 1396us) client_negotiation.cc:240] Negotiation successful
0811 03:14:27.038263 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15064,"thread_start_us":12801,"threads_started":1}
I20260811 03:14:27.040032 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:26.992026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35284 (local address 127.11.60.2:38989)
0811 03:14:27.006375 (+ 14349us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:27.006384 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:27.007269 (+ 885us) server_negotiation.cc:407] Connection header received
0811 03:14:27.007380 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:27.007390 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:27.007680 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:27.007879 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:27.008748 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.010218 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.034119 (+ 23901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.035851 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.035984 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.036215 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:27.039251 (+ 3036us) server_negotiation.cc:299] Negotiation successful
0811 03:14:27.039537 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14148,"thread_start_us":14368,"threads_started":1}
I20260811 03:14:27.044654 26299 tablet_bootstrap.cc:492] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab: Bootstrap starting.
I20260811 03:14:27.048365 26050 tablet_service.cc:1467] Processing CreateTablet for tablet 55e799f39fe144b19482001d10a3f554 (DEFAULT_TABLE table=test-table [id=c14424e2da3f4f8f82fe669399bb7456]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:14:27.049777 26299 tablet_bootstrap.cc:654] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:27.051337 26299 log.cc:824] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:27.051997 26050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e799f39fe144b19482001d10a3f554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:27.061038 26299 tablet_bootstrap.cc:492] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab: No bootstrap required, opened a new log
I20260811 03:14:27.061467 26299 ts_tablet_manager.cc:1397] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.008s
I20260811 03:14:27.072161 26299 raft_consensus.cc:348] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 } } peers { permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 } } peers { permanent_uuid: "02bd1146223e481595f632648d51f044" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38989 } }
I20260811 03:14:27.072834 26299 raft_consensus.cc:374] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:27.073004 26299 raft_consensus.cc:729] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 085e3567078b4fb9a556d1ca9fb1e0ab, State: Initialized, Role: FOLLOWER
I20260811 03:14:27.073815 26299 consensus_queue.cc:260] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 } } peers { permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 } } peers { permanent_uuid: "02bd1146223e481595f632648d51f044" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38989 } }
I20260811 03:14:27.082278 26299 ts_tablet_manager.cc:1428] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab: Time spent starting tablet: real 0.021s user 0.019s sys 0.000s
I20260811 03:14:27.082731 25962 heartbeater.cc:503] Master 127.11.60.62:41997 was elected leader, sending a full tablet report...
I20260811 03:14:27.087447 26302 tablet_bootstrap.cc:492] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: Bootstrap starting.
I20260811 03:14:27.091868 26302 tablet_bootstrap.cc:654] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:27.093672 26302 log.cc:824] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:27.095491 26123 heartbeater.cc:503] Master 127.11.60.62:41997 was elected leader, sending a full tablet report...
I20260811 03:14:27.098634 26303 tablet_bootstrap.cc:492] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: Bootstrap starting.
I20260811 03:14:27.103253 26303 tablet_bootstrap.cc:654] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:27.107450 26303 log.cc:824] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:27.115888 26303 tablet_bootstrap.cc:492] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: No bootstrap required, opened a new log
I20260811 03:14:27.115906 26302 tablet_bootstrap.cc:492] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: No bootstrap required, opened a new log
I20260811 03:14:27.116230 26303 ts_tablet_manager.cc:1397] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.004s
I20260811 03:14:27.116225 26302 ts_tablet_manager.cc:1397] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: Time spent bootstrapping tablet: real 0.032s user 0.018s sys 0.009s
I20260811 03:14:27.127498 26303 raft_consensus.cc:348] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 } } peers { permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 } } peers { permanent_uuid: "02bd1146223e481595f632648d51f044" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38989 } }
I20260811 03:14:27.127504 26302 raft_consensus.cc:348] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 } } peers { permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 } } peers { permanent_uuid: "02bd1146223e481595f632648d51f044" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38989 } }
I20260811 03:14:27.127943 26302 raft_consensus.cc:374] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:27.128132 26303 raft_consensus.cc:374] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:27.128127 26302 raft_consensus.cc:729] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52a4cabb390f4fc9b4b8995292861f68, State: Initialized, Role: FOLLOWER
I20260811 03:14:27.128293 26303 raft_consensus.cc:729] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02bd1146223e481595f632648d51f044, State: Initialized, Role: FOLLOWER
I20260811 03:14:27.128952 26303 consensus_queue.cc:260] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 } } peers { permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 } } peers { permanent_uuid: "02bd1146223e481595f632648d51f044" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38989 } }
I20260811 03:14:27.129004 26302 consensus_queue.cc:260] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 } } peers { permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 } } peers { permanent_uuid: "02bd1146223e481595f632648d51f044" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38989 } }
I20260811 03:14:27.138028 26303 ts_tablet_manager.cc:1428] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: Time spent starting tablet: real 0.022s user 0.014s sys 0.004s
W20260811 03:14:27.141782 25963 tablet.cc:2369] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:14:27.142481 26302 ts_tablet_manager.cc:1428] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: Time spent starting tablet: real 0.026s user 0.010s sys 0.015s
I20260811 03:14:27.142988 26276 heartbeater.cc:503] Master 127.11.60.62:41997 was elected leader, sending a full tablet report...
I20260811 03:14:27.149329 26307 raft_consensus.cc:482] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:14:27.149673 26307 raft_consensus.cc:504] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 } } peers { permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 } } peers { permanent_uuid: "02bd1146223e481595f632648d51f044" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38989 } }
I20260811 03:14:27.155586 26307 leader_election.cc:288] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 085e3567078b4fb9a556d1ca9fb1e0ab (127.11.60.1:46023), 52a4cabb390f4fc9b4b8995292861f68 (127.11.60.3:34643)
I20260811 03:14:27.164376 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.152893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:56301 (local address 127.11.60.1:46023)
0811 03:14:27.153088 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:27.153095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:27.156432 (+ 3337us) server_negotiation.cc:407] Connection header received
0811 03:14:27.156656 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:27.156661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:27.156773 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:27.156863 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:27.157983 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.159017 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.162029 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.163226 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.163331 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.163447 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:27.163985 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 03:14:27.164124 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 03:14:27.165462 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.152513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46023 (local address 127.11.60.2:56301)
0811 03:14:27.156299 (+ 3786us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:27.156329 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:27.156506 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:27.157333 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:27.157338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:27.157366 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:27.157796 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:27.157808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.159242 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.159247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:27.161831 (+ 2584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:27.161849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.163630 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.163636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:27.163799 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.163831 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 03:14:27.164857 (+ 1026us) client_negotiation.cc:240] Negotiation successful
0811 03:14:27.165004 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3632,"mutex_wait_us":29,"thread_start_us":3453,"threads_started":1}
I20260811 03:14:27.165575 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.156645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:44901 (local address 127.11.60.3:34643)
0811 03:14:27.156837 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:27.156843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:27.158168 (+ 1325us) server_negotiation.cc:407] Connection header received
0811 03:14:27.158360 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:27.158368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:27.158453 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:27.158542 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:27.159262 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.160382 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.163254 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.164408 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.164520 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.164619 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:27.165163 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 03:14:27.165301 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 03:14:27.166354 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.156020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:34643 (local address 127.11.60.2:44901)
0811 03:14:27.158055 (+ 2035us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:27.158074 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:27.158209 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:27.158700 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:27.158704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:27.158728 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:27.159068 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:27.159080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.160715 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.160722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:27.163067 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:27.163082 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.164818 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.164823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:27.164980 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.165009 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 03:14:27.165815 (+ 806us) client_negotiation.cc:240] Negotiation successful
0811 03:14:27.165951 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1907,"mutex_wait_us":2,"thread_start_us":816,"threads_started":1}
I20260811 03:14:27.166788 25911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e799f39fe144b19482001d10a3f554" candidate_uuid: "02bd1146223e481595f632648d51f044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" is_pre_election: true
I20260811 03:14:27.167496 25911 raft_consensus.cc:2454] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02bd1146223e481595f632648d51f044 in term 0.
I20260811 03:14:27.167763 26227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e799f39fe144b19482001d10a3f554" candidate_uuid: "02bd1146223e481595f632648d51f044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52a4cabb390f4fc9b4b8995292861f68" is_pre_election: true
I20260811 03:14:27.168295 26227 raft_consensus.cc:2454] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02bd1146223e481595f632648d51f044 in term 0.
I20260811 03:14:27.168315 26001 leader_election.cc:302] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02bd1146223e481595f632648d51f044, 085e3567078b4fb9a556d1ca9fb1e0ab; no voters:
I20260811 03:14:27.168749 26307 raft_consensus.cc:2781] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:14:27.168926 26307 raft_consensus.cc:482] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:14:27.169036 26307 raft_consensus.cc:3037] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:27.172109 26307 raft_consensus.cc:504] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 } } peers { permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 } } peers { permanent_uuid: "02bd1146223e481595f632648d51f044" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38989 } }
I20260811 03:14:27.173183 25911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e799f39fe144b19482001d10a3f554" candidate_uuid: "02bd1146223e481595f632648d51f044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab"
I20260811 03:14:27.173467 26307 leader_election.cc:288] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [CANDIDATE]: Term 1 election: Requested vote from peers 085e3567078b4fb9a556d1ca9fb1e0ab (127.11.60.1:46023), 52a4cabb390f4fc9b4b8995292861f68 (127.11.60.3:34643)
I20260811 03:14:27.173447 25911 raft_consensus.cc:3037] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:27.173964 26227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e799f39fe144b19482001d10a3f554" candidate_uuid: "02bd1146223e481595f632648d51f044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52a4cabb390f4fc9b4b8995292861f68"
I20260811 03:14:27.174170 26227 raft_consensus.cc:3037] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:27.176761 25911 raft_consensus.cc:2454] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02bd1146223e481595f632648d51f044 in term 1.
I20260811 03:14:27.177173 26001 leader_election.cc:302] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02bd1146223e481595f632648d51f044, 085e3567078b4fb9a556d1ca9fb1e0ab; no voters:
I20260811 03:14:27.177366 26227 raft_consensus.cc:2454] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02bd1146223e481595f632648d51f044 in term 1.
I20260811 03:14:27.177495 26307 raft_consensus.cc:2781] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:14:27.179719 26307 raft_consensus.cc:686] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 1 LEADER]: Becoming Leader. State: Replica: 02bd1146223e481595f632648d51f044, State: Running, Role: LEADER
I20260811 03:14:27.180323 26307 consensus_queue.cc:237] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 } } peers { permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 } } peers { permanent_uuid: "02bd1146223e481595f632648d51f044" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38989 } }
I20260811 03:14:27.185688 25743 catalog_manager.cc:5495] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02bd1146223e481595f632648d51f044 (127.11.60.2). New cstate: current_term: 1 leader_uuid: "02bd1146223e481595f632648d51f044" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02bd1146223e481595f632648d51f044" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38989 } health_report { overall_health: HEALTHY } } }
I20260811 03:14:27.231938 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.225750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35294 (local address 127.11.60.2:38989)
0811 03:14:27.225980 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:27.225984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:27.226005 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:14:27.226193 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:27.226197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:27.226394 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:27.226540 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:14:27.226986 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.227757 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.229694 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.230115 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.230213 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.230797 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:14:27.231312 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:14:27.231435 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:27.231572 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 03:14:27.231639 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 03:14:27.240692 26227 raft_consensus.cc:1261] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68 [term 1 FOLLOWER]: Refusing update from remote peer 02bd1146223e481595f632648d51f044: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:14:27.240684 25911 raft_consensus.cc:1261] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab [term 1 FOLLOWER]: Refusing update from remote peer 02bd1146223e481595f632648d51f044: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:14:27.241741 26319 consensus_queue.cc:1035] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [LEADER]: Connected to new peer: Peer: permanent_uuid: "085e3567078b4fb9a556d1ca9fb1e0ab" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 46023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:14:27.242125 26307 consensus_queue.cc:1035] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52a4cabb390f4fc9b4b8995292861f68" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 34643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:14:27.257390 26323 mvcc.cc:205] Tried to move back new op lower bound from 7317168403407073280 to 7317168403174780928. Current Snapshot: MvccSnapshot[applied={T|T < 7317168403407073280}]
I20260811 03:14:27.270433 26325 mvcc.cc:205] Tried to move back new op lower bound from 7317168403407073280 to 7317168403174780928. Current Snapshot: MvccSnapshot[applied={T|T < 7317168403407073280}]
I20260811 03:14:27.278646 25743 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:14:27.278980 25743 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:14:27.283540 25743 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9dc20f6028544508298b5b5342fa2a3: Sending DeleteTablet for 3 replicas of tablet 55e799f39fe144b19482001d10a3f554
I20260811 03:14:27.284763 25890 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e799f39fe144b19482001d10a3f554 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:14:27 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:54784
I20260811 03:14:27.284951 26200 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e799f39fe144b19482001d10a3f554 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:14:27 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:35132
I20260811 03:14:27.285590 26050 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e799f39fe144b19482001d10a3f554 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:14:27 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:35284
I20260811 03:14:27.292634 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 25786
I20260811 03:14:27.295749 26336 tablet_replica.cc:330] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: stopping tablet replica
I20260811 03:14:27.298273 26336 raft_consensus.cc:2229] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:14:27.301520 26336 raft_consensus.cc:2258] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:14:27.310467 26333 tablet_replica.cc:330] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab: stopping tablet replica
I20260811 03:14:27.310979 26333 raft_consensus.cc:2229] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:14:27.311569 26333 raft_consensus.cc:2258] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:14:27.312413 26335 tablet_replica.cc:330] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: stopping tablet replica
I20260811 03:14:27.313066 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.293707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.11.60.1:46023)
0811 03:14:27.293894 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:27.293900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:27.293936 (+ 36us) server_negotiation.cc:407] Connection header received
0811 03:14:27.294110 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:27.294118 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:27.294223 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:27.294358 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:27.296127 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.297341 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.299730 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.300573 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.300722 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.304081 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:27.304102 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:27.304125 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:27.304185 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:27.309556 (+ 5371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:27.310419 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:27.310430 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:27.310434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:27.310702 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:27.311088 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:27.311097 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:27.311100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:27.311632 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:27.312242 (+ 610us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:27.312423 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 03:14:27.312681 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 03:14:27.313050 26335 raft_consensus.cc:2229] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:14:27.313616 26335 raft_consensus.cc:2258] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:14:27.314095 25957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:27.316588 26335 ts_tablet_manager.cc:1905] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:14:27.324244 26336 ts_tablet_manager.cc:1905] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:14:27.327301 26335 ts_tablet_manager.cc:1918] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:14:27.327553 26335 log.cc:1196] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/55e799f39fe144b19482001d10a3f554
I20260811 03:14:27.328172 26335 ts_tablet_manager.cc:1939] T 55e799f39fe144b19482001d10a3f554 P 52a4cabb390f4fc9b4b8995292861f68: Deleting consensus metadata
I20260811 03:14:27.330201 25724 catalog_manager.cc:4841] TS 52a4cabb390f4fc9b4b8995292861f68 (127.11.60.3:34643): tablet 55e799f39fe144b19482001d10a3f554 (table test-table [id=c14424e2da3f4f8f82fe669399bb7456]) successfully deleted
I20260811 03:14:27.340786 26336 ts_tablet_manager.cc:1918] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:14:27.341111 26336 log.cc:1196] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/55e799f39fe144b19482001d10a3f554
I20260811 03:14:27.341804 26336 ts_tablet_manager.cc:1939] T 55e799f39fe144b19482001d10a3f554 P 02bd1146223e481595f632648d51f044: Deleting consensus metadata
I20260811 03:14:27.343642 25724 catalog_manager.cc:4841] TS 02bd1146223e481595f632648d51f044 (127.11.60.2:38989): tablet 55e799f39fe144b19482001d10a3f554 (table test-table [id=c14424e2da3f4f8f82fe669399bb7456]) successfully deleted
I20260811 03:14:27.420728 26333 ts_tablet_manager.cc:1905] T 55e799f39fe144b19482001d10a3f554 P 085e3567078b4fb9a556d1ca9fb1e0ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:14:27.423115 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 25786
I20260811 03:14:27.443706 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 25966
W20260811 03:14:27.443670 25726 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.60.1:46023 (error 108)
I20260811 03:14:27.445811 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.445255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46023 (local address 127.0.0.1:54804)
0811 03:14:27.445482 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:27.445629 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260811 03:14:27.446084 25726 catalog_manager.cc:4584] TS 085e3567078b4fb9a556d1ca9fb1e0ab (127.11.60.1:46023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55e799f39fe144b19482001d10a3f554: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46023: connect: Connection refused (error 111)
I20260811 03:14:27.460003 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.444628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35296 (local address 127.11.60.2:38989)
0811 03:14:27.444819 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:27.444825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:27.444845 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:14:27.444989 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:27.444998 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:27.445089 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:27.445193 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:27.446906 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.448168 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.449380 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.450074 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.450208 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.452881 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:27.452896 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:27.452916 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:27.452982 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:27.456853 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:27.457689 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:27.457699 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:27.457702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:27.457880 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:27.458261 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:27.458266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:27.458270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:27.458775 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:27.459091 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:27.459335 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 03:14:27.459597 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 03:14:27.460896 26118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:27.499805 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.499191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46023 (local address 127.0.0.1:54816)
0811 03:14:27.499453 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:27.499599 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 03:14:27.500999 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.500594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46023 (local address 127.0.0.1:54826)
0811 03:14:27.500746 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:27.500836 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 03:14:27.561774 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.561182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46023 (local address 127.0.0.1:54836)
0811 03:14:27.561431 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:27.561580 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 03:14:27.562964 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.562556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46023 (local address 127.0.0.1:54848)
0811 03:14:27.562726 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:27.562817 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 03:14:27.573777 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 25966
I20260811 03:14:27.595398 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 26128
I20260811 03:14:27.609197 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.596207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35140 (local address 127.11.60.3:34643)
0811 03:14:27.596439 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:27.596444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:27.596478 (+ 34us) server_negotiation.cc:407] Connection header received
0811 03:14:27.596704 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:27.596709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:27.596794 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:27.596886 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:27.598198 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.599070 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.600443 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:27.600913 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:27.601007 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:27.603763 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:27.603776 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:27.603792 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:27.603825 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:27.606698 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:27.607413 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:27.607420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:27.607422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:27.607542 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:27.607868 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:27.607871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:27.607873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:27.608222 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:27.608473 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:27.608692 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 03:14:27.608852 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 03:14:27.609870 26271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:27.643543 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.643050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46023 (local address 127.0.0.1:54858)
0811 03:14:27.643260 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:27.643352 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 03:14:27.644656 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:27.644260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:46023 (local address 127.0.0.1:54868)
0811 03:14:27.644468 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:27.644548 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 03:14:27.706674 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 26128
I20260811 03:14:27.725303 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 25703
I20260811 03:14:27.726584 25778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:27.845358 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 25703
I20260811 03:14:27.873386 11504 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 03:14:28.218722 11504 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 03:14:28.218961 11504 mini_ranger.cc:66] Stopping Ranger...
I20260811 03:14:28.564183 11504 mini_ranger.cc:68] Stopped Ranger
2026-08-11 03:14:28.564 UTC [24635] LOG: received smart shutdown request
2026-08-11 03:14:28.567 UTC [24635] LOG: background worker "logical replication launcher" (PID 24646) exited with exit code 1
2026-08-11 03:14:28.568 UTC [24640] LOG: shutting down
2026-08-11 03:14:28.759 UTC [24635] LOG: database system is shut down
2026-08-11T03:14:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54469 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:14:29 dist-test-slave-tm1g krb5kdc[26351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:14:29 dist-test-slave-tm1g krb5kdc[26351](info): set up 2 sockets
Aug 11 03:14:29 dist-test-slave-tm1g krb5kdc[26351](info): commencing operation
krb5kdc: starting...
W20260811 03:14:30.081509 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:14:30 dist-test-slave-tm1g krb5kdc[26351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:14:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:14:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:14:30.350101 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:37093
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:42165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:37093,127.11.60.61:43197,127.11.60.60:37841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:30.685459 26366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:30.685791 26366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:30.685959 26366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:30.693105 26366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:14:30.693244 26366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:30.693348 26366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:30.693457 26366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:14:30.693545 26366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:14:30.703282 26366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:42165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:37093,127.11.60.61:43197,127.11.60.60:37841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:37093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:30.704655 26366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:30.706003 26366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:30.717794 26373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:30.717840 26376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:30.717752 26374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:30.718977 26366 server_base.cc:1034] running on GCE node
I20260811 03:14:30.719715 26366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:30.721045 26366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:30.722358 26366 hybrid_clock.cc:648] HybridClock initialized: now 1786418070722264 us; error 29 us; skew 500 ppm
I20260811 03:14:30.722731 26366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:30 dist-test-slave-tm1g krb5kdc[26351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418070, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:30.731024 26366 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:14:30.732443 26366 webserver.cc:460] Webserver started at http://127.11.60.62:39491/ using document root <none> and password file <none>
I20260811 03:14:30.733139 26366 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:30.733278 26366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:30.733590 26366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:14:30.736557 26366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "41f3c9df010a440a980dfc406d7604ee"
format_stamp: "Formatted at 2026-08-11 03:14:30 on dist-test-slave-tm1g"
server_key: "f7eef6005ef499419f05a8f11bf82d16"
server_key_iv: "f0eae92ae62595d60e18c2d12b8bbe45"
server_key_version: "encryptionkey@0"
I20260811 03:14:30.737355 26366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "41f3c9df010a440a980dfc406d7604ee"
format_stamp: "Formatted at 2026-08-11 03:14:30 on dist-test-slave-tm1g"
server_key: "f7eef6005ef499419f05a8f11bf82d16"
server_key_iv: "f0eae92ae62595d60e18c2d12b8bbe45"
server_key_version: "encryptionkey@0"
I20260811 03:14:30.742501 26366 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 03:14:30.745723 26383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:30.747094 26366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:14:30.747284 26366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "41f3c9df010a440a980dfc406d7604ee"
format_stamp: "Formatted at 2026-08-11 03:14:30 on dist-test-slave-tm1g"
server_key: "f7eef6005ef499419f05a8f11bf82d16"
server_key_iv: "f0eae92ae62595d60e18c2d12b8bbe45"
server_key_version: "encryptionkey@0"
I20260811 03:14:30.747488 26366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:30.763280 26366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:30.767360 26366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:30.767643 26366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:30.786793 26366 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:37093
I20260811 03:14:30.786836 26439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:37093 every 8 connection(s)
I20260811 03:14:30.788139 26366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:14:30.791644 26440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:30.794366 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 26366
I20260811 03:14:30.794620 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:14:30.795449 11504 external_mini_cluster.cc:1428] Setting key ddc4dc2a74deb36bb52f82db31d2073c
I20260811 03:14:30.800698 26440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:14:30 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418070, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.60.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.61@KRBTEST.COM" created.
I20260811 03:14:30.826200 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:30.804386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37093 (local address 127.0.0.1:40036)
0811 03:14:30.805231 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:30.805277 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:30.808066 (+ 2789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:30.808963 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:30.808980 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:30.809758 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:30.810257 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:30.810282 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:30.811840 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:30.811848 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:30.813793 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:30.813811 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:30.814829 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:30.814840 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:30.815082 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:30.815783 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:30.815816 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:30.818496 (+ 2680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:30.821675 (+ 3179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:30.821689 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:30.821701 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:30.822109 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:30.822588 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:30.822592 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:30.822596 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:30.822759 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:30.823583 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:30.823596 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:30.823805 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 03:14:30.824149 (+ 344us) client_negotiation.cc:240] Negotiation successful
0811 03:14:30.824811 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":209,"thread_start_us":360,"threads_started":1}
I20260811 03:14:30.826174 26443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:30.805700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40036 (local address 127.11.60.62:37093)
0811 03:14:30.806426 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:30.806437 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:30.806490 (+ 53us) server_negotiation.cc:407] Connection header received
0811 03:14:30.808497 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:30.808502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:30.808717 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:30.808874 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:30.810486 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:30.811681 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:30.813944 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:30.814576 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:30.814714 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:30.818719 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:30.818727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:30.818738 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:30.818765 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:30.821542 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:30.822254 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:30.822261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:30.822263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:30.822456 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:30.822873 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:30.822878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:30.822881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:30.823440 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:30.823660 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:30.824288 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 03:14:30.824696 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":484,"thread_start_us":332,"threads_started":1}
I20260811 03:14:30.829321 26440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:14:30.830598 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:30.830095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58668)
0811 03:14:30.830282 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:30.830483 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 03:14:30.830900 26386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
I20260811 03:14:30.835609 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:30.835235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58672)
0811 03:14:30.835397 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:30.835467 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 03:14:30.835783 26440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:43197: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab.
Entry for principal kudu/127.11.60.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab.
I20260811 03:14:30.858157 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.61:43197
--webserver_interface=127.11.60.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab
--principal=kudu/127.11.60.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:42165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:37093,127.11.60.61:43197,127.11.60.60:37841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:14:30.885308 26440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } attempt: 1
I20260811 03:14:30.886577 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:30.886049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58686)
0811 03:14:30.886260 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:30.886434 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 03:14:30.887661 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:30.887288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58694)
0811 03:14:30.887434 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:30.887508 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 03:14:30.887840 26440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:43197: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
I20260811 03:14:30.956080 26440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } attempt: 2
I20260811 03:14:30.957358 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:30.956789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58702)
0811 03:14:30.957036 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:30.957174 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 03:14:30.958539 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:30.958050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58710)
0811 03:14:30.958251 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:30.958376 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 03:14:30.958730 26440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:43197: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
I20260811 03:14:31.050010 26440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } attempt: 3
I20260811 03:14:31.051455 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.050829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58718)
0811 03:14:31.051094 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.051246 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 03:14:31.052709 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.052226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58732)
0811 03:14:31.052413 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.052516 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 03:14:31.052908 26440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:43197: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
W20260811 03:14:31.195207 26448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:31.195493 26448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:31.195667 26448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 03:14:31.196206 26440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } attempt: 4
I20260811 03:14:31.197616 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.196981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58744)
0811 03:14:31.197327 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.197441 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260811 03:14:31.198894 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.198458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58760)
0811 03:14:31.198622 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.198742 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 03:14:31.199112 26440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:43197: Network error: Client connection negotiation failed: client connection to 127.11.60.61:43197: connect: Connection refused (error 111)
W20260811 03:14:31.203769 26448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:14:31.203864 26448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:31.203910 26448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:31.203969 26448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:14:31.204013 26448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:14:31.213770 26448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:42165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:37093,127.11.60.61:43197,127.11.60.60:37841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab
--principal=kudu/127.11.60.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.61:43197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.60.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:31.215116 26448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:31.216332 26448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:31.228245 26456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:31.228276 26453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:31.228379 26454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:31.231621 26448 server_base.cc:1034] running on GCE node
I20260811 03:14:31.232300 26448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:31.233578 26448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:31.234807 26448 hybrid_clock.cc:648] HybridClock initialized: now 1786418071234767 us; error 58 us; skew 500 ppm
I20260811 03:14:31.235149 26448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:31.238485 26448 init.cc:377] Logged in from keytab as kudu/127.11.60.61@KRBTEST.COM (short username kudu)
I20260811 03:14:31.240064 26448 webserver.cc:460] Webserver started at http://127.11.60.61:42023/ using document root <none> and password file <none>
I20260811 03:14:31.240780 26448 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:31.240926 26448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:31.241237 26448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:14:31.244251 26448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/data/instance:
uuid: "1ef5c8b027e34b279b8692564c896ed7"
format_stamp: "Formatted at 2026-08-11 03:14:31 on dist-test-slave-tm1g"
server_key: "e79fdca2f4e5122adce2b85ce4898719"
server_key_iv: "ec0f5a620b775f4e04b70eb98f18c3a5"
server_key_version: "encryptionkey@0"
I20260811 03:14:31.245040 26448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/wal/instance:
uuid: "1ef5c8b027e34b279b8692564c896ed7"
format_stamp: "Formatted at 2026-08-11 03:14:31 on dist-test-slave-tm1g"
server_key: "e79fdca2f4e5122adce2b85ce4898719"
server_key_iv: "ec0f5a620b775f4e04b70eb98f18c3a5"
server_key_version: "encryptionkey@0"
I20260811 03:14:31.250022 26448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:14:31.253432 26464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:31.254791 26448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 03:14:31.254973 26448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/wal
uuid: "1ef5c8b027e34b279b8692564c896ed7"
format_stamp: "Formatted at 2026-08-11 03:14:31 on dist-test-slave-tm1g"
server_key: "e79fdca2f4e5122adce2b85ce4898719"
server_key_iv: "ec0f5a620b775f4e04b70eb98f18c3a5"
server_key_version: "encryptionkey@0"
I20260811 03:14:31.255173 26448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:31.273276 26448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:31.277065 26448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:31.277352 26448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:31.296182 26448 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.61:43197
I20260811 03:14:31.296221 26521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.61:43197 every 8 connection(s)
I20260811 03:14:31.297586 26448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
I20260811 03:14:31.301215 26522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:31.303761 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 26448
I20260811 03:14:31.303959 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-1/wal/instance
I20260811 03:14:31.304560 11504 external_mini_cluster.cc:1428] Setting key cdb5f688decf3800f6c89276cea3ad33
I20260811 03:14:31.310504 26522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.60.60@KRBTEST.COM; defaulting to no policy
I20260811 03:14:31.333410 26443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.313887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.11.60.62:37093)
0811 03:14:31.314108 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.314115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.315382 (+ 1267us) server_negotiation.cc:407] Connection header received
0811 03:14:31.317436 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.317442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.317511 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.317602 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.319605 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.320290 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.322787 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.323195 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.323290 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.327529 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.327534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.327537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.327568 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.330347 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.331072 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.331077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.331082 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.331205 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.331633 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.331637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.331641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.331914 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.332073 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.332781 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.333010 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:14:31.334293 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.314359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37093 (local address 127.0.0.1:40038)
0811 03:14:31.315211 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.315256 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:31.317157 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:31.317847 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:31.317860 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:31.318654 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:31.319406 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.319432 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.320452 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.320460 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.322648 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.322665 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.323426 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.323436 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.323772 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.324529 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:31.324561 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:31.327266 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:31.330502 (+ 3236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.330510 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.330525 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.330944 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.331318 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.331322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.331326 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.331513 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.332046 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:31.332054 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:31.332400 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 03:14:31.332672 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 03:14:31.333157 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":388,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.60@KRBTEST.COM" created.
I20260811 03:14:31.336182 26522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
I20260811 03:14:31.360266 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.337052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58770)
0811 03:14:31.337256 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.337282 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:31.337405 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:31.345133 (+ 7728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:31.345141 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:31.345675 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:31.345824 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.345833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.347925 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.347929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.348738 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.348749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.350012 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.350016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.350225 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.350879 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:31.350896 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:31.353148 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:31.357591 (+ 4443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.357594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.357597 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.357988 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.358478 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.358482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.358484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.358638 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.359474 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:31.359478 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:31.359619 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 03:14:31.359730 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 03:14:31.360004 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":35}
I20260811 03:14:31.360757 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.337024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58770 (local address 127.11.60.61:43197)
0811 03:14:31.344705 (+ 7681us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.344711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.344736 (+ 25us) server_negotiation.cc:407] Connection header received
0811 03:14:31.344778 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.344784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.344955 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.345082 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.345989 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.347788 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.348883 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.349853 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.349992 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.353383 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.353399 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.353418 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.353460 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.357447 (+ 3987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.358149 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.358160 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.358165 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.358349 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.358801 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.358809 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.358814 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.359310 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.359519 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.359946 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.360228 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":7536,"thread_start_us":7411,"threads_started":1}
I20260811 03:14:31.363399 26522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:14:31.364445 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.364012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35820)
0811 03:14:31.364187 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.364285 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 03:14:31.364765 26467 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
I20260811 03:14:31.366611 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.366192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35832)
0811 03:14:31.366414 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.366516 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 03:14:31.366871 26522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:37841: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab.
Entry for principal kudu/127.11.60.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab.
I20260811 03:14:31.369751 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.60:37841
--webserver_interface=127.11.60.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab
--principal=kudu/127.11.60.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:42165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:37093,127.11.60.61:43197,127.11.60.60:37841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:14:31.416221 26522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } attempt: 1
I20260811 03:14:31.417523 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.417057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35838)
0811 03:14:31.417265 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.417368 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 03:14:31.418527 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.418151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35844)
0811 03:14:31.418350 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.418424 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 03:14:31.418733 26522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:37841: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
I20260811 03:14:31.486965 26522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } attempt: 2
I20260811 03:14:31.488124 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.487663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35846)
0811 03:14:31.487870 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.487965 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 03:14:31.489095 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.488754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35862)
0811 03:14:31.488900 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.488973 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 03:14:31.489302 26522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:37841: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
I20260811 03:14:31.498406 26440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } attempt: 5
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418070, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
I20260811 03:14:31.513290 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.499055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58780)
0811 03:14:31.499283 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.499306 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:31.499442 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:31.499754 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:31.499759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:31.500334 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:31.500572 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.500586 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.501571 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.501579 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.503107 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.503124 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.503890 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.503898 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.504131 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.504929 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:31.504959 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:31.507378 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:31.510865 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.510870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.510875 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.511247 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.511612 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.511617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.511621 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.511761 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.512293 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:31.512298 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:31.512459 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 03:14:31.512609 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 03:14:31.512896 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 03:14:31.513286 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.499295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58780 (local address 127.11.60.61:43197)
0811 03:14:31.499463 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.499467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.499488 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:14:31.499544 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.499547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.499603 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.499684 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.500737 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.501418 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.503246 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.503708 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.503803 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.507577 (+ 3774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.507582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.507585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.507617 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.510722 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.511364 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.511371 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.511375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.511501 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.511869 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.511876 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.511879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.512172 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.512329 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.512720 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.512922 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 03:14:31.514662 26440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:14:31.515635 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.515249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35870)
0811 03:14:31.515412 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.515500 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 03:14:31.516513 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.516194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35876)
0811 03:14:31.516329 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.516414 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 03:14:31.516731 26440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:37841: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
I20260811 03:14:31.567978 26440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } attempt: 1
I20260811 03:14:31.569412 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.568692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35888)
0811 03:14:31.568997 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.569156 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 03:14:31.570739 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.570228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35904)
0811 03:14:31.570459 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.570571 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 03:14:31.570993 26440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:37841: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
I20260811 03:14:31.580513 26522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } attempt: 3
I20260811 03:14:31.581560 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.581118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35908)
0811 03:14:31.581302 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.581402 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 03:14:31.582712 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.582266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35920)
0811 03:14:31.582530 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.582604 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
W20260811 03:14:31.583036 26522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:37841: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
I20260811 03:14:31.639240 26440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } attempt: 2
I20260811 03:14:31.640661 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.640013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35930)
0811 03:14:31.640320 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.640411 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260811 03:14:31.641726 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.641361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35942)
0811 03:14:31.641555 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.641625 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 03:14:31.641932 26440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:37841: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
W20260811 03:14:31.712761 26530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:31.713110 26530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:31.713266 26530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:31.720824 26530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:14:31.720971 26530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:31.721079 26530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:31.721184 26530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:14:31.721274 26530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:14:31.726269 26522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } attempt: 4
I20260811 03:14:31.727551 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.727056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35946)
0811 03:14:31.727282 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.727371 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 03:14:31.728523 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.728205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35958)
0811 03:14:31.728346 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.728422 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 03:14:31.728716 26522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:37841: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
I20260811 03:14:31.731873 26530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:42165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:37093,127.11.60.61:43197,127.11.60.60:37841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab
--principal=kudu/127.11.60.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.60:37841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.60.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:31.733230 26530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:31.734643 26530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 03:14:31.748231 26440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } attempt: 3
W20260811 03:14:31.749616 26538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:31.749779 26535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:31.749909 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.749162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35972)
0811 03:14:31.749588 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.749693 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
W20260811 03:14:31.750272 26536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:31.750810 26530 server_base.cc:1034] running on GCE node
I20260811 03:14:31.750974 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.750596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:35978)
0811 03:14:31.750773 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.750858 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 03:14:31.751159 26440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:37841: Network error: Client connection negotiation failed: client connection to 127.11.60.60:37841: connect: Connection refused (error 111)
I20260811 03:14:31.751740 26530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:31.753302 26530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:31.756644 26530 hybrid_clock.cc:648] HybridClock initialized: now 1786418071756595 us; error 52 us; skew 500 ppm
I20260811 03:14:31.757033 26530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:31.760389 26530 init.cc:377] Logged in from keytab as kudu/127.11.60.60@KRBTEST.COM (short username kudu)
I20260811 03:14:31.761864 26530 webserver.cc:460] Webserver started at http://127.11.60.60:36581/ using document root <none> and password file <none>
I20260811 03:14:31.762679 26530 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:31.762826 26530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:31.763140 26530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:14:31.766238 26530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/data/instance:
uuid: "394351856cd7415cb06cebbda01d387d"
format_stamp: "Formatted at 2026-08-11 03:14:31 on dist-test-slave-tm1g"
server_key: "21b7fb39572cba3cf78f43b0f4b0ba32"
server_key_iv: "863a8be2e6bdadf0bca9ab061f0c08fc"
server_key_version: "encryptionkey@0"
I20260811 03:14:31.767135 26530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/wal/instance:
uuid: "394351856cd7415cb06cebbda01d387d"
format_stamp: "Formatted at 2026-08-11 03:14:31 on dist-test-slave-tm1g"
server_key: "21b7fb39572cba3cf78f43b0f4b0ba32"
server_key_iv: "863a8be2e6bdadf0bca9ab061f0c08fc"
server_key_version: "encryptionkey@0"
I20260811 03:14:31.772249 26530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:14:31.775775 26547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:31.777068 26530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:14:31.777249 26530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/wal
uuid: "394351856cd7415cb06cebbda01d387d"
format_stamp: "Formatted at 2026-08-11 03:14:31 on dist-test-slave-tm1g"
server_key: "21b7fb39572cba3cf78f43b0f4b0ba32"
server_key_iv: "863a8be2e6bdadf0bca9ab061f0c08fc"
server_key_version: "encryptionkey@0"
I20260811 03:14:31.777442 26530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:31.792388 26530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:31.796687 26530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:31.796981 26530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:31.814208 26530 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.60:37841
I20260811 03:14:31.814240 26603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.60:37841 every 8 connection(s)
I20260811 03:14:31.815586 26530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
I20260811 03:14:31.815999 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 26530
I20260811 03:14:31.816211 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417858274425-11504-0/minicluster-data/master-2/wal/instance
I20260811 03:14:31.816931 11504 external_mini_cluster.cc:1428] Setting key 0b9dd1137d069016dda5699ade9a9018
I20260811 03:14:31.819913 26604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:31.826876 26604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:31.839383 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.818902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.11.60.60:37841)
0811 03:14:31.819677 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.819695 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.819734 (+ 39us) server_negotiation.cc:407] Connection header received
0811 03:14:31.821362 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.821382 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.821727 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.822095 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.823587 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.824918 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.826245 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.826960 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.827161 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.831813 (+ 4652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.831819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.831828 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.831865 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.834911 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.835816 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.835821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.835827 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.835989 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.836438 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.836443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.836446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.837160 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.837433 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.837844 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.838224 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":348,"threads_started":1}
I20260811 03:14:31.844197 26443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.829231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40054 (local address 127.11.60.62:37093)
0811 03:14:31.829473 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.829479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.829997 (+ 518us) server_negotiation.cc:407] Connection header received
0811 03:14:31.830198 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.830207 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.830294 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.830429 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.831515 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.832316 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.833708 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.834150 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.834280 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.838008 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.838015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.838020 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.838063 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.840961 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.841800 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.841807 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.841810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.841985 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.842560 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.842565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.842568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.842965 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.843207 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.843638 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.843845 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 03:14:31.844206 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.829097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37093 (local address 127.0.0.1:40054)
0811 03:14:31.829829 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.829885 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:31.830063 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:31.830506 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:31.830510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:31.831094 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:31.831350 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.831358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.832497 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.832501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.833554 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.833568 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.834376 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.834386 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.834563 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.835164 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:31.835196 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:31.837807 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:31.841110 (+ 3303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.841119 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.841125 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.841680 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.842102 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.842105 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.842107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.842376 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.843135 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:31.843145 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:31.843315 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 03:14:31.843492 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 03:14:31.843797 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":319,"threads_started":1}
I20260811 03:14:31.845489 26604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
I20260811 03:14:31.858130 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.846203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:43197 (local address 127.0.0.1:58790)
0811 03:14:31.846375 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.846397 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:31.846496 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:31.846815 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:31.846819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:31.847156 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:31.847275 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.847283 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.848212 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.848219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.849067 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.849077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.849764 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.849770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.849963 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.850551 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:31.850566 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:31.852874 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:31.855771 (+ 2897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.855777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.855781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.856181 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.856563 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.856568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.856572 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.856718 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.857212 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:31.857217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:31.857356 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 03:14:31.857510 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 03:14:31.857770 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 03:14:31.858173 26609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.842566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40064 (local address 127.11.60.62:37093)
0811 03:14:31.843180 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.843184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.843203 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:14:31.843240 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.843243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.843292 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.843405 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.845030 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.845941 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.847489 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.847899 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.848038 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.852442 (+ 4404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.852449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.852454 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.852491 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.855558 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.856208 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.856215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.856218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.856340 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.856735 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.856740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.856743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.857020 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.857304 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.857587 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.857807 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":277,"threads_started":1}
I20260811 03:14:31.858574 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.846245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58790 (local address 127.11.60.61:43197)
0811 03:14:31.846519 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.846527 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.846548 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:14:31.846634 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.846638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.846692 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.846794 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.847419 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.848075 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.849209 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.849618 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.849749 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.853061 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.853066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.853069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.853099 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.855646 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.856305 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.856310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.856313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.856450 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.856843 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.856847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.856849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.857098 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.857247 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.857600 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.857815 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260811 03:14:31.859563 26604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
I20260811 03:14:31.873608 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.860208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:36004)
0811 03:14:31.860340 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.860360 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:31.860447 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:31.860726 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:31.860731 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:31.861134 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:31.861253 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.861261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.862560 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.862563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.863828 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.863842 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.864559 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.864564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.864687 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.865228 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:31.865250 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:31.867613 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:31.871202 (+ 3589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.871206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.871208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.871550 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.871939 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.871942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.871944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.872061 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.872561 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:31.872565 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:31.872694 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 03:14:31.872890 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 03:14:31.873199 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 03:14:31.873798 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.860304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36004 (local address 127.11.60.60:37841)
0811 03:14:31.860433 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.860438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.860454 (+ 16us) server_negotiation.cc:407] Connection header received
0811 03:14:31.860553 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.860557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.860617 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.860709 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.861390 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.862424 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.863968 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.864395 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.864514 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.867784 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.867791 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.867796 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.867834 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.871073 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.871674 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.871681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.871690 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.871804 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.872164 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.872171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.872174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.872432 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.872582 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.872998 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.873348 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":36}
I20260811 03:14:31.875607 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.859831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.11.60.61:43197)
0811 03:14:31.859983 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.859988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.860007 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:14:31.860221 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.860228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.860296 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.860388 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.861690 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.862733 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.864402 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.864988 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.865130 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.869679 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.869687 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.869693 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.869734 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.872654 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.873457 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.873463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.873467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.873629 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.874087 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.874093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.874099 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.874448 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.874663 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.875042 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.875261 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 03:14:31.879199 26604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d: Bootstrap starting.
I20260811 03:14:31.880357 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:14:31.882594 26604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:31.883747 26604 log.cc:824] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:31.886626 26604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d: No bootstrap required, opened a new log
I20260811 03:14:31.900856 26604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } }
I20260811 03:14:31.901369 26604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:31.901482 26604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394351856cd7415cb06cebbda01d387d, State: Initialized, Role: FOLLOWER
I20260811 03:14:31.902773 26604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } }
I20260811 03:14:31.905718 26622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } } }
I20260811 03:14:31.906148 26622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:14:31.907369 26604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:14:31.912177 26609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.888468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40066 (local address 127.11.60.62:37093)
0811 03:14:31.888694 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.888701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.889653 (+ 952us) server_negotiation.cc:407] Connection header received
0811 03:14:31.889956 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.889964 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.890069 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.890148 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.893215 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.894670 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.897518 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.898450 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.898573 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.903102 (+ 4529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.903109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.903113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.903156 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.908007 (+ 4851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.909050 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.909056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.909060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.909222 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.909690 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.909699 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.909702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.910119 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.910405 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.911509 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.911764 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 03:14:31.919785 26634 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:14:31.919914 26634 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:14:31.922742 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.888888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58814 (local address 127.11.60.61:43197)
0811 03:14:31.889110 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.889118 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.900159 (+ 11041us) server_negotiation.cc:407] Connection header received
0811 03:14:31.900361 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.900366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.900439 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.900555 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.902699 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.903793 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.906007 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.906704 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.906802 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.915003 (+ 8201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.915010 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.915014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.915063 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.918954 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.919814 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.919821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.919827 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.919995 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.920439 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.920446 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.920449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.920842 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.921087 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.922091 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.922355 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 03:14:31.927651 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.890156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.11.60.60:37841)
0811 03:14:31.890415 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.890424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.902520 (+ 12096us) server_negotiation.cc:407] Connection header received
0811 03:14:31.902769 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.902775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.902854 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.902923 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.905123 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.906294 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.908404 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.909201 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.909347 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.912747 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.912753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.912761 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.912807 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.919352 (+ 6545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.925017 (+ 5665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.925023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.925027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.925212 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.925711 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.925717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.925720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.926109 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.926396 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.926981 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.927216 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 03:14:31.928373 26440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } attempt: 4
I20260811 03:14:31.916275 26530 master_runner.cc:186] Error getting master registration for 127.11.60.62:37093: OK, instance_id { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" instance_seqno: 1786418070781440 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:14:31.928669 26530 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 11 03:14:31 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418070, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
I20260811 03:14:31.944258 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.928909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:36030)
0811 03:14:31.929124 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:31.929147 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:31.929261 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:31.929783 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:31.929789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:31.930276 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:31.930503 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.930518 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.931561 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.931568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.933083 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:31.933099 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.933830 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.933837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:31.934029 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.935025 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:31.935048 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:31.937514 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:31.941819 (+ 4305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.941823 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.941826 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.942179 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.942600 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:31.942603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:31.942606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:31.942730 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:31.943267 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:31.943272 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:31.943433 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 03:14:31.943579 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 03:14:31.943868 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 03:14:31.944280 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:31.929299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36030 (local address 127.11.60.60:37841)
0811 03:14:31.929513 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:31.929523 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:31.929537 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:14:31.929580 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:31.929583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:31.929632 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:31.929702 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:31.930643 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.931426 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.933216 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:31.933677 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:31.933773 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:31.937690 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:31.937697 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:31.937702 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:31.937743 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:31.941672 (+ 3929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.942324 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.942330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.942334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.942464 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:31.942841 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:31.942847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:31.942850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:31.943146 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:31.943301 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:31.943676 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 03:14:31.943896 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":107}
I20260811 03:14:31.949105 26440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee: Bootstrap starting.
I20260811 03:14:31.952294 26440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:31.953282 26440 log.cc:824] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:31.956499 26440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee: No bootstrap required, opened a new log
I20260811 03:14:31.963979 26440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } }
I20260811 03:14:31.964337 26440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:31.964469 26440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f3c9df010a440a980dfc406d7604ee, State: Initialized, Role: FOLLOWER
I20260811 03:14:31.965086 26440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } }
I20260811 03:14:31.967643 26637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } } }
I20260811 03:14:31.968117 26637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:14:31.968441 26440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [sys.catalog]: configured and running, proceeding with master startup.
W20260811 03:14:31.980393 26648 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:14:31.980621 26648 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:14:31.977843 26366 master_runner.cc:186] Error getting master registration for 127.11.60.61:43197: OK, instance_id { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" instance_seqno: 1786418071290728 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:14:31.987129 26366 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 03:14:32.000977 26622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:14:32.001361 26622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } }
I20260811 03:14:32.003238 26622 leader_election.cc:288] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41f3c9df010a440a980dfc406d7604ee (127.11.60.62:37093), 1ef5c8b027e34b279b8692564c896ed7 (127.11.60.61:43197)
I20260811 03:14:32.004834 26413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "394351856cd7415cb06cebbda01d387d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f3c9df010a440a980dfc406d7604ee" is_pre_election: true
I20260811 03:14:32.005532 26413 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 394351856cd7415cb06cebbda01d387d in term 0.
I20260811 03:14:32.006197 26551 leader_election.cc:302] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 394351856cd7415cb06cebbda01d387d, 41f3c9df010a440a980dfc406d7604ee; no voters:
I20260811 03:14:32.006645 26622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:14:32.006807 26622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:14:32.006927 26622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:32.010510 26622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } }
I20260811 03:14:32.011989 26413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "394351856cd7415cb06cebbda01d387d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f3c9df010a440a980dfc406d7604ee"
I20260811 03:14:32.012215 26413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:32.012585 26622 leader_election.cc:288] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [CANDIDATE]: Term 1 election: Requested vote from peers 41f3c9df010a440a980dfc406d7604ee (127.11.60.62:37093), 1ef5c8b027e34b279b8692564c896ed7 (127.11.60.61:43197)
I20260811 03:14:32.015866 26413 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 394351856cd7415cb06cebbda01d387d in term 1.
I20260811 03:14:32.016288 26551 leader_election.cc:302] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 394351856cd7415cb06cebbda01d387d, 41f3c9df010a440a980dfc406d7604ee; no voters:
I20260811 03:14:32.016619 26622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:14:32.017344 26622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [term 1 LEADER]: Becoming Leader. State: Replica: 394351856cd7415cb06cebbda01d387d, State: Running, Role: LEADER
I20260811 03:14:32.018031 26622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } }
W20260811 03:14:32.020402 26389 tablet.cc:2369] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:14:32.023499 26650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 394351856cd7415cb06cebbda01d387d. Latest consensus state: current_term: 1 leader_uuid: "394351856cd7415cb06cebbda01d387d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } } }
I20260811 03:14:32.023932 26650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:32.025449 26652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:14:32.027906 26522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } attempt: 5
I20260811 03:14:32.028365 26652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 03:14:32 dist-test-slave-tm1g krb5kdc[26351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
W20260811 03:14:32.049180 26553 tablet.cc:2369] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:14:32.056893 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:32.029211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36042 (local address 127.11.60.60:37841)
0811 03:14:32.029424 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:32.029431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:32.029449 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:14:32.029503 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:32.029508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:32.029578 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:32.029680 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:32.031046 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:32.032254 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:32.034717 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:32.035697 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:32.035860 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:32.045538 (+ 9678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:32.045548 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:32.045557 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:32.045610 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:32.050350 (+ 4740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:32.051482 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:32.051488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:32.051491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:32.051750 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:32.052667 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:32.052673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:32.052676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:32.053345 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:32.053565 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:32.054408 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 03:14:32.054867 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 03:14:32.057003 26413 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [term 1 FOLLOWER]: Refusing update from remote peer 394351856cd7415cb06cebbda01d387d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:14:32.057950 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:32.028469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:37841 (local address 127.0.0.1:36042)
0811 03:14:32.028851 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:32.028873 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:32.029002 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:32.029951 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:32.029957 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:32.030594 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:32.030812 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:32.030825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:32.032472 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:32.032477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:32.034455 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:32.034473 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:32.036256 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:32.036265 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:32.036714 (+ 449us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:32.038403 (+ 1689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:32.038429 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:32.045124 (+ 6695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:32.050564 (+ 5440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:32.050573 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:32.050580 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:32.051306 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:32.052003 (+ 697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:32.052009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:32.052015 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:32.052469 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:32.053683 (+ 1214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:32.053689 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:32.053929 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 03:14:32.055072 (+ 1143us) client_negotiation.cc:240] Negotiation successful
0811 03:14:32.056717 (+ 1645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260811 03:14:32.058804 26650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [LEADER]: Connected to new peer: Peer: permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:14:32.063732 26522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7: Bootstrap starting.
I20260811 03:14:32.069300 26522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:32.071836 26522 log.cc:824] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:32.087273 26522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7: No bootstrap required, opened a new log
I20260811 03:14:32.095538 26637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 394351856cd7415cb06cebbda01d387d. Latest consensus state: current_term: 1 leader_uuid: "394351856cd7415cb06cebbda01d387d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } } }
I20260811 03:14:32.095929 26637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:14:32.105850 26654 mvcc.cc:205] Tried to move back new op lower bound from 7317168423128510464 to 7317168422998065152. Current Snapshot: MvccSnapshot[applied={T|T < 7317168423128510464}]
I20260811 03:14:32.112111 26522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } }
I20260811 03:14:32.113499 26522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:32.113792 26522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef5c8b027e34b279b8692564c896ed7, State: Initialized, Role: FOLLOWER
I20260811 03:14:32.115222 26522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } }
I20260811 03:14:32.118654 26622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "394351856cd7415cb06cebbda01d387d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } } }
I20260811 03:14:32.119072 26622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:32.121182 26637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "394351856cd7415cb06cebbda01d387d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } } }
I20260811 03:14:32.121465 26637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:14:32.123751 26659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } } }
I20260811 03:14:32.124208 26659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:14:32.126238 26522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:14:32.126518 26497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "394351856cd7415cb06cebbda01d387d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef5c8b027e34b279b8692564c896ed7" is_pre_election: true
I20260811 03:14:32.127146 26497 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 394351856cd7415cb06cebbda01d387d in term 0.
I20260811 03:14:32.127543 26496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "394351856cd7415cb06cebbda01d387d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef5c8b027e34b279b8692564c896ed7"
I20260811 03:14:32.127766 26496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:32.132315 26496 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 394351856cd7415cb06cebbda01d387d in term 1.
I20260811 03:14:32.133318 26495 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [term 1 FOLLOWER]: Refusing update from remote peer 394351856cd7415cb06cebbda01d387d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:14:32.134752 26652 catalog_manager.cc:1347] Generated new cluster ID: 02876b63fea54e929f6bd866fb5bcae4
I20260811 03:14:32.134859 26652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:14:32.134438 26650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260811 03:14:32.183944 26659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 394351856cd7415cb06cebbda01d387d. Latest consensus state: current_term: 1 leader_uuid: "394351856cd7415cb06cebbda01d387d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } } }
I20260811 03:14:32.184459 26659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:14:32.183967 26665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "394351856cd7415cb06cebbda01d387d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } } }
I20260811 03:14:32.184681 26665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:14:32.222607 26650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "394351856cd7415cb06cebbda01d387d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f3c9df010a440a980dfc406d7604ee" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37093 } } peers { permanent_uuid: "1ef5c8b027e34b279b8692564c896ed7" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 43197 } } peers { permanent_uuid: "394351856cd7415cb06cebbda01d387d" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 37841 } } }
I20260811 03:14:32.222892 26650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:32.224172 26652 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:14:32.226802 26652 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:14:32.255577 26652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 394351856cd7415cb06cebbda01d387d: Generated new TSK 0
I20260811 03:14:32.256881 26652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:14:32.262291 26676 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 03:14:32.982944 26648 catalog_manager.cc:1259] Loaded cluster ID: 02876b63fea54e929f6bd866fb5bcae4
I20260811 03:14:32.983086 26648 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee: loading cluster ID for follower catalog manager: success
I20260811 03:14:32.985630 26648 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee: acquiring CA information for follower catalog manager: success
I20260811 03:14:32.987367 26648 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 41f3c9df010a440a980dfc406d7604ee: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:14:33.266821 26676 catalog_manager.cc:1259] Loaded cluster ID: 02876b63fea54e929f6bd866fb5bcae4
I20260811 03:14:33.266973 26676 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7: loading cluster ID for follower catalog manager: success
I20260811 03:14:33.269840 26676 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7: acquiring CA information for follower catalog manager: success
I20260811 03:14:33.271958 26676 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1ef5c8b027e34b279b8692564c896ed7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:14:33.358203 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 26366
I20260811 03:14:33.359424 26435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:33.505920 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 26366
I20260811 03:14:33.531654 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 26448
I20260811 03:14:33.533022 26517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:33.652661 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 26448
I20260811 03:14:33.674510 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 26530
I20260811 03:14:33.675657 26599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:33.790619 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 26530
2026-08-11T03:14:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4893 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:14:33 dist-test-slave-tm1g krb5kdc[26686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:14:33 dist-test-slave-tm1g krb5kdc[26686](info): set up 2 sockets
Aug 11 03:14:33 dist-test-slave-tm1g krb5kdc[26686](info): commencing operation
krb5kdc: starting...
W20260811 03:14:34.975097 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.007s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:14:35 dist-test-slave-tm1g krb5kdc[26686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:14:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:14:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:14:35.262072 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42257
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:42257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:35.610065 26699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:35.610487 26699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:35.610649 26699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:35.618204 26699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:14:35.618369 26699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:35.618474 26699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:35.618573 26699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:14:35.618649 26699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:14:35.628738 26699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:42257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:35.630132 26699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:35.631559 26699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:35.643527 26704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:35.643573 26708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:35.643597 26705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:35.644277 26699 server_base.cc:1034] running on GCE node
I20260811 03:14:35.645061 26699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:35.646548 26699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:35.647778 26699 hybrid_clock.cc:648] HybridClock initialized: now 1786418075647744 us; error 50 us; skew 500 ppm
I20260811 03:14:35.648139 26699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:35 dist-test-slave-tm1g krb5kdc[26686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418075, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:35.651388 26699 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:14:35.652942 26699 webserver.cc:460] Webserver started at http://127.11.60.62:45659/ using document root <none> and password file <none>
I20260811 03:14:35.653661 26699 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:35.653811 26699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:35.654143 26699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:14:35.657212 26699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "c940fe65ccb14b1a9281522003918f40"
format_stamp: "Formatted at 2026-08-11 03:14:35 on dist-test-slave-tm1g"
server_key: "9fe4647c58543fd4cf39744d42068555"
server_key_iv: "e8e3b460248b14e585b924587b8c285b"
server_key_version: "encryptionkey@0"
I20260811 03:14:35.657984 26699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "c940fe65ccb14b1a9281522003918f40"
format_stamp: "Formatted at 2026-08-11 03:14:35 on dist-test-slave-tm1g"
server_key: "9fe4647c58543fd4cf39744d42068555"
server_key_iv: "e8e3b460248b14e585b924587b8c285b"
server_key_version: "encryptionkey@0"
I20260811 03:14:35.663281 26699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 03:14:35.666762 26716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:35.668051 26699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 03:14:35.668238 26699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "c940fe65ccb14b1a9281522003918f40"
format_stamp: "Formatted at 2026-08-11 03:14:35 on dist-test-slave-tm1g"
server_key: "9fe4647c58543fd4cf39744d42068555"
server_key_iv: "e8e3b460248b14e585b924587b8c285b"
server_key_version: "encryptionkey@0"
I20260811 03:14:35.668444 26699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:35.690886 26699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:35.694965 26699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:35.695209 26699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:35.714088 26699 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:42257
I20260811 03:14:35.714123 26771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:42257 every 8 connection(s)
I20260811 03:14:35.715513 26699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:14:35.718976 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 26699
I20260811 03:14:35.719218 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:14:35.719278 26772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:35.719954 11504 external_mini_cluster.cc:1428] Setting key b5ce4e56727e15fee5135e67682caf7f
I20260811 03:14:35.733258 26772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40: Bootstrap starting.
I20260811 03:14:35.736697 26772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:35.737766 26772 log.cc:824] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:35.740869 26772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40: No bootstrap required, opened a new log
Aug 11 03:14:35 dist-test-slave-tm1g krb5kdc[26686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:35.751919 26772 raft_consensus.cc:348] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c940fe65ccb14b1a9281522003918f40" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42257 } }
I20260811 03:14:35.752386 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:35.721788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35440 (local address 127.11.60.62:42257)
0811 03:14:35.722685 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:35.722696 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:35.722736 (+ 40us) server_negotiation.cc:407] Connection header received
0811 03:14:35.724256 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:35.724277 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:35.724656 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:35.724980 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:35.727026 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:35.728310 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:35.730222 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:35.730983 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:35.731124 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:35.742802 (+ 11678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:35.742817 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:35.742843 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:35.742938 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:35.747685 (+ 4747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:35.748461 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:35.748470 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:35.748479 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:35.748705 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:35.749073 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:35.749078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:35.749081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:35.749858 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:35.750181 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:35.750542 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 03:14:35.751018 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":456,"threads_started":1}
I20260811 03:14:35.752417 26772 raft_consensus.cc:374] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:35.752517 26772 raft_consensus.cc:729] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c940fe65ccb14b1a9281522003918f40, State: Initialized, Role: FOLLOWER
I20260811 03:14:35.753131 26772 consensus_queue.cc:260] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c940fe65ccb14b1a9281522003918f40" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42257 } }
I20260811 03:14:35.753346 26772 raft_consensus.cc:388] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:14:35.753445 26772 raft_consensus.cc:482] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:14:35.753604 26772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:35.756290 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:14:35.756232 26772 raft_consensus.cc:504] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c940fe65ccb14b1a9281522003918f40" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42257 } }
I20260811 03:14:35.756781 26772 leader_election.cc:302] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c940fe65ccb14b1a9281522003918f40; no voters:
I20260811 03:14:35.757427 26772 leader_election.cc:288] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:14:35.757536 26778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:14:35.758152 26778 raft_consensus.cc:686] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [term 1 LEADER]: Becoming Leader. State: Replica: c940fe65ccb14b1a9281522003918f40, State: Running, Role: LEADER
I20260811 03:14:35.758765 26778 consensus_queue.cc:237] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c940fe65ccb14b1a9281522003918f40" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42257 } }
I20260811 03:14:35.760555 26778 sys_catalog.cc:455] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c940fe65ccb14b1a9281522003918f40. Latest consensus state: current_term: 1 leader_uuid: "c940fe65ccb14b1a9281522003918f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c940fe65ccb14b1a9281522003918f40" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42257 } } }
I20260811 03:14:35.760846 26778 sys_catalog.cc:458] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:35.762140 26780 sys_catalog.cc:455] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c940fe65ccb14b1a9281522003918f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c940fe65ccb14b1a9281522003918f40" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42257 } } }
I20260811 03:14:35.762677 26780 sys_catalog.cc:458] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:35.765208 26772 sys_catalog.cc:564] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:14:35.765447 26782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:14:35.770604 26782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:14:35.780140 26782 catalog_manager.cc:1347] Generated new cluster ID: 50fd54e84a3040e48786fe580207a8ef
I20260811 03:14:35.780272 26782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:14:35.787457 26782 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:14:35.788748 26782 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:14:35.799958 26782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c940fe65ccb14b1a9281522003918f40: Generated new TSK 0
I20260811 03:14:35.800765 26782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:14:35.818138 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 26699
I20260811 03:14:35.819401 26767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:35.911830 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 26699
2026-08-11T03:14:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2116 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:14:36 dist-test-slave-tm1g krb5kdc[26803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:14:36 dist-test-slave-tm1g krb5kdc[26803](info): set up 2 sockets
Aug 11 03:14:36 dist-test-slave-tm1g krb5kdc[26803](info): commencing operation
krb5kdc: starting...
W20260811 03:14:37.088742 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:14:37 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:14:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:14:37.378551 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:44637
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:44637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:37.736635 26818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:37.736989 26818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:37.737170 26818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:37.748064 26818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:14:37.748232 26818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:37.748332 26818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:37.748421 26818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:14:37.748528 26818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:14:37.759454 26818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:44637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:44637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:37.760960 26818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:37.762425 26818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:37.775632 26825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:37.775569 26827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:37.776078 26824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:37.776922 26818 server_base.cc:1034] running on GCE node
I20260811 03:14:37.777776 26818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:37.779441 26818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:37.780699 26818 hybrid_clock.cc:648] HybridClock initialized: now 1786418077780655 us; error 54 us; skew 500 ppm
I20260811 03:14:37.781073 26818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:37 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418077, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:37.784425 26818 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:14:37.785961 26818 webserver.cc:460] Webserver started at http://127.11.60.62:37809/ using document root <none> and password file <none>
I20260811 03:14:37.786749 26818 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:37.786860 26818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:37.787122 26818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:14:37.790374 26818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "83b4d74dc47d4aa3afbd665961fa8c22"
format_stamp: "Formatted at 2026-08-11 03:14:37 on dist-test-slave-tm1g"
server_key: "4b43dd7811c224536e0041cdc5f51239"
server_key_iv: "3596fdd301bcd483c2243b94ad1730d9"
server_key_version: "encryptionkey@0"
I20260811 03:14:37.791137 26818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "83b4d74dc47d4aa3afbd665961fa8c22"
format_stamp: "Formatted at 2026-08-11 03:14:37 on dist-test-slave-tm1g"
server_key: "4b43dd7811c224536e0041cdc5f51239"
server_key_iv: "3596fdd301bcd483c2243b94ad1730d9"
server_key_version: "encryptionkey@0"
I20260811 03:14:37.796705 26818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:14:37.800298 26834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:37.801764 26818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:14:37.801982 26818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "83b4d74dc47d4aa3afbd665961fa8c22"
format_stamp: "Formatted at 2026-08-11 03:14:37 on dist-test-slave-tm1g"
server_key: "4b43dd7811c224536e0041cdc5f51239"
server_key_iv: "3596fdd301bcd483c2243b94ad1730d9"
server_key_version: "encryptionkey@0"
I20260811 03:14:37.802223 26818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:37.828562 26818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:37.833355 26818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:37.833689 26818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:37.852702 26818 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:44637
I20260811 03:14:37.852744 26891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:44637 every 8 connection(s)
I20260811 03:14:37.854132 26818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:14:37.857950 26892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:37.858757 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 26818
I20260811 03:14:37.858987 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:14:37.859743 11504 external_mini_cluster.cc:1428] Setting key 6169f7523be80e79442a6be7efdf3813
I20260811 03:14:37.872614 26892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Bootstrap starting.
Aug 11 03:14:37 dist-test-slave-tm1g krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:37.876632 26892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:37.877969 26892 log.cc:824] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:37.881634 26892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: No bootstrap required, opened a new log
I20260811 03:14:37.886122 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:37.861911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37808 (local address 127.11.60.62:44637)
0811 03:14:37.862798 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:37.862813 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:37.862865 (+ 52us) server_negotiation.cc:407] Connection header received
0811 03:14:37.864772 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:37.864803 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:37.865336 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:37.865819 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:37.867339 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:37.869423 (+ 2084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:37.870936 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:37.871966 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:37.872158 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:37.877157 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:37.877171 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:37.877190 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:37.877240 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:37.881280 (+ 4040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:37.882141 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:37.882151 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:37.882165 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:37.882413 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:37.882801 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:37.882806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:37.882810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:37.883513 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:37.883858 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:37.884219 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 03:14:37.884703 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":364,"threads_started":1}
I20260811 03:14:37.891566 26892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:37.891906 26892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:37.891971 26892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b4d74dc47d4aa3afbd665961fa8c22, State: Initialized, Role: FOLLOWER
I20260811 03:14:37.892532 26892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:37.892836 26892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:14:37.892968 26892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:14:37.893108 26892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:37.895439 26892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:37.896026 26892 leader_election.cc:302] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83b4d74dc47d4aa3afbd665961fa8c22; no voters:
I20260811 03:14:37.896634 26892 leader_election.cc:288] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:14:37.896802 26897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:14:37.897610 26897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 1 LEADER]: Becoming Leader. State: Replica: 83b4d74dc47d4aa3afbd665961fa8c22, State: Running, Role: LEADER
I20260811 03:14:37.898164 26897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:37.899008 26892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:14:37.901372 26899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83b4d74dc47d4aa3afbd665961fa8c22. Latest consensus state: current_term: 1 leader_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } } }
I20260811 03:14:37.901754 26899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:37.901566 26898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } } }
I20260811 03:14:37.901998 26898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:37.904157 26906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:14:37.908978 26906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:14:37.917598 26906 catalog_manager.cc:1347] Generated new cluster ID: fc8d61ceca364394897d72f83ad16995
I20260811 03:14:37.917716 26906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:14:37.945195 26906 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:14:37.947429 26906 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:14:37.955785 26906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Generated new TSK 0
I20260811 03:14:37.956634 26906 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:14:38.029778 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44637
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:38.384110 26919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:38.384464 26919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:38.384652 26919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:38.395263 26919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:38.395399 26919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:38.396154 26919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:14:38.407876 26919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:38.409282 26919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:38.410923 26919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:38.425170 26925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:38.425290 26924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:38.425570 26928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:38.425988 26919 server_base.cc:1034] running on GCE node
I20260811 03:14:38.426759 26919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:38.427802 26919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:38.429039 26919 hybrid_clock.cc:648] HybridClock initialized: now 1786418078428984 us; error 49 us; skew 500 ppm
I20260811 03:14:38.429414 26919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:38 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418078, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:38.432916 26919 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:14:38.434454 26919 webserver.cc:460] Webserver started at http://127.11.60.1:46627/ using document root <none> and password file <none>
I20260811 03:14:38.435170 26919 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:38.435283 26919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:38.435557 26919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:14:38.438632 26919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "9baa0849eddf431098946b6e1e31a1c3"
format_stamp: "Formatted at 2026-08-11 03:14:38 on dist-test-slave-tm1g"
server_key: "a7a840d45be373d4cc59ffd9c54d9da3"
server_key_iv: "4c4da85a6cdb7538ef518ac1a539d5ba"
server_key_version: "encryptionkey@0"
I20260811 03:14:38.439421 26919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "9baa0849eddf431098946b6e1e31a1c3"
format_stamp: "Formatted at 2026-08-11 03:14:38 on dist-test-slave-tm1g"
server_key: "a7a840d45be373d4cc59ffd9c54d9da3"
server_key_iv: "4c4da85a6cdb7538ef518ac1a539d5ba"
server_key_version: "encryptionkey@0"
I20260811 03:14:38.444918 26919 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 03:14:38.448846 26935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:38.450263 26919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:14:38.450513 26919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "9baa0849eddf431098946b6e1e31a1c3"
format_stamp: "Formatted at 2026-08-11 03:14:38 on dist-test-slave-tm1g"
server_key: "a7a840d45be373d4cc59ffd9c54d9da3"
server_key_iv: "4c4da85a6cdb7538ef518ac1a539d5ba"
server_key_version: "encryptionkey@0"
I20260811 03:14:38.450758 26919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:38.470023 26919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:38.474824 26919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:38.475080 26919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:38.476065 26919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:38.477439 26919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:38.477527 26919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:38.477617 26919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:38.477660 26919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:38.513314 26919 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:34941
I20260811 03:14:38.513366 27053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:34941 every 8 connection(s)
I20260811 03:14:38.514719 26919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:14:38.519340 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 26919
I20260811 03:14:38.519596 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:14:38.520264 11504 external_mini_cluster.cc:1428] Setting key 8d826afe71c959fee673d5f3ef67b789
Aug 11 03:14:38 dist-test-slave-tm1g krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418078, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:38.535739 27056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:38.517350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:35707 (local address 127.11.60.62:44637)
0811 03:14:38.517856 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:38.517861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:38.518750 (+ 889us) server_negotiation.cc:407] Connection header received
0811 03:14:38.520768 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:38.520773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:38.520877 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:38.521009 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:38.522925 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:38.523636 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:38.525239 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:38.525657 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:38.525797 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:38.529643 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:38.529648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:38.529651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:38.529685 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:38.532400 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:38.533267 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:38.533272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:38.533274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:38.533416 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:38.533938 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:38.533942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:38.533944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:38.534218 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:38.534445 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:38.535263 (+ 818us) server_negotiation.cc:299] Negotiation successful
0811 03:14:38.535473 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":274,"threads_started":1}
I20260811 03:14:38.537322 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:38.517781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:35707)
0811 03:14:38.518571 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:38.518626 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:38.520480 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:38.521264 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:38.521273 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:38.522041 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:38.522739 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:38.522758 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:38.523803 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:38.523809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:38.525078 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:38.525091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:38.525910 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:38.525917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:38.526163 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:38.526921 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:38.526949 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:38.529394 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:38.532572 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:38.532581 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:38.532599 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:38.533112 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:38.533531 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:38.533539 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:38.533544 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:38.533808 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:38.534375 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:38.534385 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:38.534846 (+ 461us) client_negotiation.cc:769] Sending connection context
0811 03:14:38.535168 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 03:14:38.535766 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":289,"threads_started":1}
I20260811 03:14:38.539053 27054 heartbeater.cc:347] Connected to a master server at 127.11.60.62:44637
I20260811 03:14:38.539378 27054 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:38.540194 27054 heartbeater.cc:511] Master 127.11.60.62:44637 requested a full tablet report, sending...
I20260811 03:14:38.542497 26852 ts_manager.cc:194] Registered new tserver with Master: 9baa0849eddf431098946b6e1e31a1c3 (127.11.60.1:34941)
I20260811 03:14:38.543994 26852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:35707
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:14:38.585573 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44637
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:38.943259 27060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:38.943624 27060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:38.943797 27060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:38.952719 27060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:38.952891 27060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:38.953088 27060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:14:38.964147 27060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:38.965649 27060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:38.967187 27060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:38.981231 27065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:38.981210 27066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:38.981592 27068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:38.982133 27060 server_base.cc:1034] running on GCE node
I20260811 03:14:38.982946 27060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:38.983968 27060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:38.985177 27060 hybrid_clock.cc:648] HybridClock initialized: now 1786418078985132 us; error 36 us; skew 500 ppm
I20260811 03:14:38.985559 27060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:38 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418078, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:38.988905 27060 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:14:38.990658 27060 webserver.cc:460] Webserver started at http://127.11.60.2:36011/ using document root <none> and password file <none>
I20260811 03:14:38.991423 27060 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:38.991571 27060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:38.991907 27060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:14:38.995078 27060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "f3560e8dbb3b45b786b624046bab9a7f"
format_stamp: "Formatted at 2026-08-11 03:14:38 on dist-test-slave-tm1g"
server_key: "cb8d970429a687a7f069a634a74d60bf"
server_key_iv: "06b51eb00e34653d9e0e2844c458a5c0"
server_key_version: "encryptionkey@0"
I20260811 03:14:38.995935 27060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "f3560e8dbb3b45b786b624046bab9a7f"
format_stamp: "Formatted at 2026-08-11 03:14:38 on dist-test-slave-tm1g"
server_key: "cb8d970429a687a7f069a634a74d60bf"
server_key_iv: "06b51eb00e34653d9e0e2844c458a5c0"
server_key_version: "encryptionkey@0"
I20260811 03:14:39.001588 27060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 03:14:39.005460 27078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:39.007211 27060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:14:39.007377 27060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "f3560e8dbb3b45b786b624046bab9a7f"
format_stamp: "Formatted at 2026-08-11 03:14:38 on dist-test-slave-tm1g"
server_key: "cb8d970429a687a7f069a634a74d60bf"
server_key_iv: "06b51eb00e34653d9e0e2844c458a5c0"
server_key_version: "encryptionkey@0"
I20260811 03:14:39.007575 27060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:39.023234 27060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:39.027788 27060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:39.028047 27060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:39.029004 27060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:39.030536 27060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:39.030612 27060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:39.030701 27060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:39.030751 27060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:39.066633 27060 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:45407
I20260811 03:14:39.066702 27203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:45407 every 8 connection(s)
I20260811 03:14:39.068043 27060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:14:39.077118 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 27060
I20260811 03:14:39.077351 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:14:39.077993 11504 external_mini_cluster.cc:1428] Setting key e1a7bd2e038cad8dda438c1e8d674a95
Aug 11 03:14:39 dist-test-slave-tm1g krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418078, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:39.091447 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.070901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:48275 (local address 127.11.60.62:44637)
0811 03:14:39.071426 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:39.071433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:39.072386 (+ 953us) server_negotiation.cc:407] Connection header received
0811 03:14:39.074243 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:39.074248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:39.074395 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:39.074561 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:39.076094 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.076886 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.079193 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.079693 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.079840 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.084756 (+ 4916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:39.084762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:39.084766 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:39.084804 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:39.087685 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:39.088727 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:39.088731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:39.088734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:39.088875 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:39.089467 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:39.089472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:39.089474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:39.089812 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:39.089953 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:39.090951 (+ 998us) server_negotiation.cc:299] Negotiation successful
0811 03:14:39.091141 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":385,"thread_start_us":244,"threads_started":1}
I20260811 03:14:39.092756 27208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.071392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:48275)
0811 03:14:39.072212 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:39.072259 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:39.073958 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:39.074778 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:39.074786 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:39.075313 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:39.075905 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:39.075921 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.077058 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.077064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:39.079025 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:39.079038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.079951 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.079959 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:39.080284 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.081143 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:39.081180 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:39.084445 (+ 3265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:39.087857 (+ 3412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:39.087870 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:39.087893 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:39.088590 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:39.088999 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:39.089006 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:39.089012 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:39.089297 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:39.089963 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:39.089974 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:39.090519 (+ 545us) client_negotiation.cc:769] Sending connection context
0811 03:14:39.090863 (+ 344us) client_negotiation.cc:240] Negotiation successful
0811 03:14:39.091518 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":329,"threads_started":1}
I20260811 03:14:39.094436 27204 heartbeater.cc:347] Connected to a master server at 127.11.60.62:44637
I20260811 03:14:39.094794 27204 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:39.095805 27204 heartbeater.cc:511] Master 127.11.60.62:44637 requested a full tablet report, sending...
I20260811 03:14:39.097635 26852 ts_manager.cc:194] Registered new tserver with Master: f3560e8dbb3b45b786b624046bab9a7f (127.11.60.2:45407)
I20260811 03:14:39.098654 26852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:48275
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:14:39.150020 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44637
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:39.504966 27211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:39.505276 27211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:39.505437 27211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:39.513146 27211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:39.513260 27211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:39.513445 27211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:14:39.523624 27211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:44637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:39.525000 27211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:39.526527 27211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:39.540643 27217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:39.540993 27219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:39.541389 27216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:39.541651 27211 server_base.cc:1034] running on GCE node
I20260811 03:14:39.542439 27211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:39.543505 27211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:39.544752 27211 hybrid_clock.cc:648] HybridClock initialized: now 1786418079544696 us; error 52 us; skew 500 ppm
I20260811 03:14:39.545114 27211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:14:39.547288 27054 heartbeater.cc:503] Master 127.11.60.62:44637 was elected leader, sending a full tablet report...
Aug 11 03:14:39 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418079, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:39.548820 27211 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:14:39.550737 27211 webserver.cc:460] Webserver started at http://127.11.60.3:42529/ using document root <none> and password file <none>
I20260811 03:14:39.551509 27211 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:39.551628 27211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:39.551889 27211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:14:39.555222 27211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "4d26616a9f79423b9d389bb0f55f6245"
format_stamp: "Formatted at 2026-08-11 03:14:39 on dist-test-slave-tm1g"
server_key: "47b1b2f43483da40e8f31db0f4e80948"
server_key_iv: "55192ff832bbcabc46c76e880725e9cd"
server_key_version: "encryptionkey@0"
I20260811 03:14:39.556035 27211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "4d26616a9f79423b9d389bb0f55f6245"
format_stamp: "Formatted at 2026-08-11 03:14:39 on dist-test-slave-tm1g"
server_key: "47b1b2f43483da40e8f31db0f4e80948"
server_key_iv: "55192ff832bbcabc46c76e880725e9cd"
server_key_version: "encryptionkey@0"
I20260811 03:14:39.561422 27211 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 03:14:39.565066 27227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:39.566604 27211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:14:39.566748 27211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "4d26616a9f79423b9d389bb0f55f6245"
format_stamp: "Formatted at 2026-08-11 03:14:39 on dist-test-slave-tm1g"
server_key: "47b1b2f43483da40e8f31db0f4e80948"
server_key_iv: "55192ff832bbcabc46c76e880725e9cd"
server_key_version: "encryptionkey@0"
I20260811 03:14:39.566931 27211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:39.584852 27211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:39.589622 27211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:39.589888 27211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:39.590927 27211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:39.592573 27211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:39.592656 27211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:39.592748 27211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:39.592795 27211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:39.631263 27211 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:37485
I20260811 03:14:39.631331 27351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:37485 every 8 connection(s)
I20260811 03:14:39.632640 27211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:14:39.640825 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 27211
I20260811 03:14:39.641057 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:14:39.641518 11504 external_mini_cluster.cc:1428] Setting key 6d9b98de1ea9f06ac2d9379adec22362
Aug 11 03:14:39 dist-test-slave-tm1g krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418079, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:39.653417 27355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.635362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:60061 (local address 127.11.60.62:44637)
0811 03:14:39.635887 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:39.635892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:39.636728 (+ 836us) server_negotiation.cc:407] Connection header received
0811 03:14:39.638628 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:39.638634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:39.638725 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:39.638844 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:39.640453 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.641450 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.643379 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.643880 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.644006 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.647667 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:39.647672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:39.647676 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:39.647711 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:39.650435 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:39.651237 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:39.651242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:39.651244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:39.651372 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:39.651804 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:39.651807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:39.651809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:39.652127 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:39.652244 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:39.652942 (+ 698us) server_negotiation.cc:299] Negotiation successful
0811 03:14:39.653137 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":273,"threads_started":1}
I20260811 03:14:39.654580 27357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.635845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:60061)
0811 03:14:39.636561 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:39.636606 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:39.638293 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:39.639096 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:39.639104 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:39.639616 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:39.640265 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:39.640283 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.641638 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.641645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:39.643229 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:39.643242 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.644128 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.644133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:39.644341 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.645120 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:39.645149 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:39.647446 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:39.650576 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:39.650583 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:39.650599 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:39.651101 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:39.651472 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:39.651477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:39.651480 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:39.651666 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:39.652241 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:39.652247 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:39.652591 (+ 344us) client_negotiation.cc:769] Sending connection context
0811 03:14:39.652843 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 03:14:39.653265 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":284,"threads_started":1}
I20260811 03:14:39.656255 27352 heartbeater.cc:347] Connected to a master server at 127.11.60.62:44637
I20260811 03:14:39.656652 27352 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:39.657681 27352 heartbeater.cc:511] Master 127.11.60.62:44637 requested a full tablet report, sending...
I20260811 03:14:39.659282 26852 ts_manager.cc:194] Registered new tserver with Master: 4d26616a9f79423b9d389bb0f55f6245 (127.11.60.3:37485)
I20260811 03:14:39.660162 26852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:60061
I20260811 03:14:39.670253 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:14:39.687883 27355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.675195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56140 (local address 127.11.60.62:44637)
0811 03:14:39.675448 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:39.675458 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:39.675795 (+ 337us) server_negotiation.cc:407] Connection header received
0811 03:14:39.675939 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:39.675944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:39.676011 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:39.676099 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:39.677547 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.678335 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.679636 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.680035 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.680126 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.682696 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:39.682701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:39.682704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:39.682741 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:39.685275 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:39.685985 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:39.685989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:39.685991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:39.686123 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:39.686490 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:39.686493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:39.686495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:39.686841 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:39.687039 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:39.687357 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 03:14:39.687564 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 03:14:39.692162 26852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:14:39.699441 26852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:14:39.737109 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.713778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.11.60.1:34941)
0811 03:14:39.725057 (+ 11279us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:39.725069 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:39.726831 (+ 1762us) server_negotiation.cc:407] Connection header received
0811 03:14:39.726972 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:39.726984 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:39.727357 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:39.727605 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:39.728956 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.730790 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.733970 (+ 3180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.735597 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.735715 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.735992 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:39.736557 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 03:14:39.736740 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11029,"thread_start_us":11298,"threads_started":1}
I20260811 03:14:39.737176 27369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.713378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34941 (local address 127.0.0.1:39478)
0811 03:14:39.726104 (+ 12726us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:39.726167 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:39.726545 (+ 378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:39.728048 (+ 1503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:39.728061 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:39.728114 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:39.728680 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:39.728698 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.730959 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.730964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:39.733742 (+ 2778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:39.733760 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.736087 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.736101 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:39.736320 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.736360 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 03:14:39.736513 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 03:14:39.736727 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12332,"mutex_wait_us":318,"thread_start_us":17722,"threads_started":1}
I20260811 03:14:39.739068 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.714086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34304 (local address 127.11.60.3:37485)
0811 03:14:39.724034 (+ 9948us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:39.724047 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:39.730000 (+ 5953us) server_negotiation.cc:407] Connection header received
0811 03:14:39.730149 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:39.730158 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:39.730454 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:39.731548 (+ 1094us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:39.731829 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.733515 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.736297 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.737814 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.737960 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.738196 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:39.738523 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 03:14:39.738704 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9670,"thread_start_us":9459,"threads_started":1}
I20260811 03:14:39.739724 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.713717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:37485 (local address 127.0.0.1:34304)
0811 03:14:39.729852 (+ 16135us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:39.729881 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:39.730008 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:39.730685 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:39.730689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:39.730713 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:39.731274 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:39.731288 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.733698 (+ 2410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.733704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:39.736115 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:39.736128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.738048 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.738054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:39.738252 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.738288 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 03:14:39.738438 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 03:14:39.738583 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15973,"mutex_wait_us":6,"thread_start_us":11824,"threads_started":1}
I20260811 03:14:39.742515 27368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.716970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:45407 (local address 127.0.0.1:34528)
0811 03:14:39.730686 (+ 13716us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:39.730704 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:39.730833 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:39.735030 (+ 4197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:39.735036 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:39.735067 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:39.735516 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:39.735529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.737589 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.737594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:39.739881 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:39.739894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.741671 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.741678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:39.741854 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.741886 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 03:14:39.741979 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 03:14:39.742127 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13600,"thread_start_us":13116,"threads_started":1}
I20260811 03:14:39.743242 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:39.717655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.11.60.2:45407)
0811 03:14:39.734113 (+ 16458us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:39.734125 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:39.734153 (+ 28us) server_negotiation.cc:407] Connection header received
0811 03:14:39.734257 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:39.734264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:39.734559 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:39.734758 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:39.735732 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.737343 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.740064 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:39.741173 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:39.741308 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:39.741528 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:39.742733 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0811 03:14:39.742914 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16253,"thread_start_us":16951,"threads_started":1}
I20260811 03:14:39.744324 27281 tablet_service.cc:1467] Processing CreateTablet for tablet 739fddf63e6645f0aaf946c1b1c9920f (DEFAULT_TABLE table=test-table [id=c1cba100f9414cff84c7a34cdd8774c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:14:39.744694 26986 tablet_service.cc:1467] Processing CreateTablet for tablet 739fddf63e6645f0aaf946c1b1c9920f (DEFAULT_TABLE table=test-table [id=c1cba100f9414cff84c7a34cdd8774c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:14:39.746526 27281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739fddf63e6645f0aaf946c1b1c9920f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:39.747108 26986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739fddf63e6645f0aaf946c1b1c9920f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:39.747733 27130 tablet_service.cc:1467] Processing CreateTablet for tablet 739fddf63e6645f0aaf946c1b1c9920f (DEFAULT_TABLE table=test-table [id=c1cba100f9414cff84c7a34cdd8774c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:14:39.749133 27130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739fddf63e6645f0aaf946c1b1c9920f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:39.761278 27374 tablet_bootstrap.cc:492] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: Bootstrap starting.
I20260811 03:14:39.763856 27376 tablet_bootstrap.cc:492] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: Bootstrap starting.
I20260811 03:14:39.764750 27375 tablet_bootstrap.cc:492] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: Bootstrap starting.
I20260811 03:14:39.766466 27374 tablet_bootstrap.cc:654] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:39.767961 27374 log.cc:824] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:39.768251 27376 tablet_bootstrap.cc:654] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:39.768736 27375 tablet_bootstrap.cc:654] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:39.769644 27376 log.cc:824] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:39.769765 27375 log.cc:824] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:39.771874 27374 tablet_bootstrap.cc:492] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: No bootstrap required, opened a new log
I20260811 03:14:39.772200 27374 ts_tablet_manager.cc:1397] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260811 03:14:39.772991 27375 tablet_bootstrap.cc:492] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: No bootstrap required, opened a new log
I20260811 03:14:39.773258 27376 tablet_bootstrap.cc:492] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: No bootstrap required, opened a new log
I20260811 03:14:39.773265 27375 ts_tablet_manager.cc:1397] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 03:14:39.773564 27376 ts_tablet_manager.cc:1397] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 03:14:39.781137 27375 raft_consensus.cc:348] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } }
I20260811 03:14:39.781585 27375 raft_consensus.cc:374] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:39.781690 27375 raft_consensus.cc:729] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3560e8dbb3b45b786b624046bab9a7f, State: Initialized, Role: FOLLOWER
I20260811 03:14:39.782414 27375 consensus_queue.cc:260] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } }
I20260811 03:14:39.783643 27204 heartbeater.cc:503] Master 127.11.60.62:44637 was elected leader, sending a full tablet report...
I20260811 03:14:39.784405 27375 ts_tablet_manager.cc:1428] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 03:14:39.784633 27374 raft_consensus.cc:348] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } }
I20260811 03:14:39.785166 27374 raft_consensus.cc:374] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:39.784911 27376 raft_consensus.cc:348] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } }
I20260811 03:14:39.785293 27374 raft_consensus.cc:729] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9baa0849eddf431098946b6e1e31a1c3, State: Initialized, Role: FOLLOWER
I20260811 03:14:39.785377 27376 raft_consensus.cc:374] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:39.785475 27376 raft_consensus.cc:729] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d26616a9f79423b9d389bb0f55f6245, State: Initialized, Role: FOLLOWER
I20260811 03:14:39.786118 27376 consensus_queue.cc:260] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } }
I20260811 03:14:39.786837 27374 consensus_queue.cc:260] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } }
I20260811 03:14:39.790386 27352 heartbeater.cc:503] Master 127.11.60.62:44637 was elected leader, sending a full tablet report...
I20260811 03:14:39.790601 27374 ts_tablet_manager.cc:1428] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: Time spent starting tablet: real 0.018s user 0.003s sys 0.015s
I20260811 03:14:39.790988 27376 ts_tablet_manager.cc:1428] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
W20260811 03:14:39.820220 27206 tablet.cc:2369] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:14:39.884402 27353 tablet.cc:2369] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:14:40.017486 27055 tablet.cc:2369] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:14:40.035559 27380 raft_consensus.cc:482] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:14:40.035887 27380 raft_consensus.cc:504] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } }
I20260811 03:14:40.037808 27380 leader_election.cc:288] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9baa0849eddf431098946b6e1e31a1c3 (127.11.60.1:34941), 4d26616a9f79423b9d389bb0f55f6245 (127.11.60.3:37485)
I20260811 03:14:40.044822 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:40.038223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:36401 (local address 127.11.60.1:34941)
0811 03:14:40.038504 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:40.038512 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:40.039268 (+ 756us) server_negotiation.cc:407] Connection header received
0811 03:14:40.039534 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:40.039538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:40.039601 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:40.039703 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:40.040276 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.040999 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.042980 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.043791 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.043959 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:40.044079 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:40.044414 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 03:14:40.044548 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 03:14:40.044947 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:40.038469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:42707 (local address 127.11.60.3:37485)
0811 03:14:40.038749 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:40.038758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:40.039441 (+ 683us) server_negotiation.cc:407] Connection header received
0811 03:14:40.039653 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:40.039657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:40.039716 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:40.039817 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:40.040484 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.041218 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.043159 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.043970 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.044131 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:40.044241 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:40.044548 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 03:14:40.044677 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 03:14:40.045050 27384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:40.038249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:37485 (local address 127.11.60.2:42707)
0811 03:14:40.039318 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:40.039339 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:40.039497 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:40.039838 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:40.039843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:40.039884 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:40.040330 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:40.040338 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.041378 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.041383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:40.042990 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:40.043003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.044194 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.044200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:40.044385 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:40.044408 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 03:14:40.044515 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 03:14:40.044641 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":44,"thread_start_us":753,"threads_started":1}
I20260811 03:14:40.045456 27383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:40.038164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34941 (local address 127.11.60.2:36401)
0811 03:14:40.039137 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:40.039154 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:40.039343 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:40.039724 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:40.039729 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:40.039752 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:40.040126 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:40.040135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.041144 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.041151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:40.042799 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:40.042813 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.044043 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.044049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:40.044226 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:40.044252 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 03:14:40.044376 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 03:14:40.044518 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":561,"threads_started":1}
I20260811 03:14:40.046233 27302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739fddf63e6645f0aaf946c1b1c9920f" candidate_uuid: "f3560e8dbb3b45b786b624046bab9a7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d26616a9f79423b9d389bb0f55f6245" is_pre_election: true
I20260811 03:14:40.046582 27007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739fddf63e6645f0aaf946c1b1c9920f" candidate_uuid: "f3560e8dbb3b45b786b624046bab9a7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9baa0849eddf431098946b6e1e31a1c3" is_pre_election: true
I20260811 03:14:40.046938 27302 raft_consensus.cc:2454] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3560e8dbb3b45b786b624046bab9a7f in term 0.
I20260811 03:14:40.047125 27007 raft_consensus.cc:2454] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3560e8dbb3b45b786b624046bab9a7f in term 0.
I20260811 03:14:40.047753 27083 leader_election.cc:302] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d26616a9f79423b9d389bb0f55f6245, f3560e8dbb3b45b786b624046bab9a7f; no voters:
I20260811 03:14:40.048267 27380 raft_consensus.cc:2781] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:14:40.048449 27380 raft_consensus.cc:482] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:14:40.048569 27380 raft_consensus.cc:3037] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:40.051004 27380 raft_consensus.cc:504] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } }
I20260811 03:14:40.051656 27380 leader_election.cc:288] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [CANDIDATE]: Term 1 election: Requested vote from peers 9baa0849eddf431098946b6e1e31a1c3 (127.11.60.1:34941), 4d26616a9f79423b9d389bb0f55f6245 (127.11.60.3:37485)
I20260811 03:14:40.052239 27007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739fddf63e6645f0aaf946c1b1c9920f" candidate_uuid: "f3560e8dbb3b45b786b624046bab9a7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9baa0849eddf431098946b6e1e31a1c3"
I20260811 03:14:40.052286 27302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739fddf63e6645f0aaf946c1b1c9920f" candidate_uuid: "f3560e8dbb3b45b786b624046bab9a7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d26616a9f79423b9d389bb0f55f6245"
I20260811 03:14:40.052471 27007 raft_consensus.cc:3037] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:40.052508 27302 raft_consensus.cc:3037] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:40.054962 27007 raft_consensus.cc:2454] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3560e8dbb3b45b786b624046bab9a7f in term 1.
I20260811 03:14:40.055048 27302 raft_consensus.cc:2454] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3560e8dbb3b45b786b624046bab9a7f in term 1.
I20260811 03:14:40.055545 27081 leader_election.cc:302] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9baa0849eddf431098946b6e1e31a1c3, f3560e8dbb3b45b786b624046bab9a7f; no voters:
I20260811 03:14:40.055917 27380 raft_consensus.cc:2781] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:14:40.056457 27380 raft_consensus.cc:686] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 1 LEADER]: Becoming Leader. State: Replica: f3560e8dbb3b45b786b624046bab9a7f, State: Running, Role: LEADER
I20260811 03:14:40.056973 27380 consensus_queue.cc:237] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } }
I20260811 03:14:40.062232 26851 catalog_manager.cc:5495] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f reported cstate change: term changed from 0 to 1, leader changed from <none> to f3560e8dbb3b45b786b624046bab9a7f (127.11.60.2). New cstate: current_term: 1 leader_uuid: "f3560e8dbb3b45b786b624046bab9a7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } health_report { overall_health: HEALTHY } } }
I20260811 03:14:40.118286 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:40.112084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.11.60.2:45407)
0811 03:14:40.112315 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:40.112320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:40.112340 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:14:40.112577 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:40.112581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:40.112684 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:40.112859 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:14:40.113299 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.114122 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.115658 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.116124 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.116234 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:40.116824 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:14:40.117480 (+ 656us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:14:40.117607 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:40.117814 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 03:14:40.117938 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 03:14:40.127768 27302 raft_consensus.cc:1261] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245 [term 1 FOLLOWER]: Refusing update from remote peer f3560e8dbb3b45b786b624046bab9a7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:14:40.127765 27007 raft_consensus.cc:1261] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3 [term 1 FOLLOWER]: Refusing update from remote peer f3560e8dbb3b45b786b624046bab9a7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:14:40.128850 27385 consensus_queue.cc:1035] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:14:40.129271 27380 consensus_queue.cc:1035] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:14:40.146488 27389 mvcc.cc:205] Tried to move back new op lower bound from 7317168456191729664 to 7317168455917572096. Current Snapshot: MvccSnapshot[applied={T|T < 7317168456191729664}]
I20260811 03:14:40.159672 26850 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:14:40.159996 26850 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:14:40.164248 26850 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Sending DeleteTablet for 3 replicas of tablet 739fddf63e6645f0aaf946c1b1c9920f
I20260811 03:14:40.165652 27281 tablet_service.cc:1514] Processing DeleteTablet for tablet 739fddf63e6645f0aaf946c1b1c9920f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:14:40 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:34304
I20260811 03:14:40.165660 26986 tablet_service.cc:1514] Processing DeleteTablet for tablet 739fddf63e6645f0aaf946c1b1c9920f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:14:40 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:39478
I20260811 03:14:40.165887 27130 tablet_service.cc:1514] Processing DeleteTablet for tablet 739fddf63e6645f0aaf946c1b1c9920f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:14:40 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:34528
I20260811 03:14:40.166159 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 26919
I20260811 03:14:40.178380 27400 tablet_replica.cc:330] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: stopping tablet replica
I20260811 03:14:40.178580 27399 tablet_replica.cc:330] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: stopping tablet replica
I20260811 03:14:40.178983 27399 raft_consensus.cc:2229] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:14:40.178999 27400 raft_consensus.cc:2229] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:14:40.179586 27400 raft_consensus.cc:2258] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:14:40.179585 27399 raft_consensus.cc:2258] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:14:40.182068 27399 ts_tablet_manager.cc:1905] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:14:40.183097 27400 ts_tablet_manager.cc:1905] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:14:40.189586 27399 ts_tablet_manager.cc:1918] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:14:40.189791 27399 log.cc:1196] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/739fddf63e6645f0aaf946c1b1c9920f
I20260811 03:14:40.190248 27401 tablet_replica.cc:330] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: stopping tablet replica
I20260811 03:14:40.190392 27399 ts_tablet_manager.cc:1939] T 739fddf63e6645f0aaf946c1b1c9920f P 4d26616a9f79423b9d389bb0f55f6245: Deleting consensus metadata
I20260811 03:14:40.190910 27401 raft_consensus.cc:2229] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:14:40.191660 27401 raft_consensus.cc:2258] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:14:40.192483 26838 catalog_manager.cc:4841] TS 4d26616a9f79423b9d389bb0f55f6245 (127.11.60.3:37485): tablet 739fddf63e6645f0aaf946c1b1c9920f (table test-table [id=c1cba100f9414cff84c7a34cdd8774c8]) successfully deleted
I20260811 03:14:40.193163 27400 ts_tablet_manager.cc:1918] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:14:40.193372 27400 log.cc:1196] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/739fddf63e6645f0aaf946c1b1c9920f
I20260811 03:14:40.194092 27400 ts_tablet_manager.cc:1939] T 739fddf63e6645f0aaf946c1b1c9920f P 9baa0849eddf431098946b6e1e31a1c3: Deleting consensus metadata
I20260811 03:14:40.194453 27401 ts_tablet_manager.cc:1905] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:14:40.196246 26837 catalog_manager.cc:4841] TS 9baa0849eddf431098946b6e1e31a1c3 (127.11.60.1:34941): tablet 739fddf63e6645f0aaf946c1b1c9920f (table test-table [id=c1cba100f9414cff84c7a34cdd8774c8]) successfully deleted
Aug 11 03:14:40 dist-test-slave-tm1g krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:14:40.202627 27401 ts_tablet_manager.cc:1918] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:14:40.202821 27401 log.cc:1196] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/739fddf63e6645f0aaf946c1b1c9920f
I20260811 03:14:40.203390 27401 ts_tablet_manager.cc:1939] T 739fddf63e6645f0aaf946c1b1c9920f P f3560e8dbb3b45b786b624046bab9a7f: Deleting consensus metadata
I20260811 03:14:40.205291 26835 catalog_manager.cc:4841] TS f3560e8dbb3b45b786b624046bab9a7f (127.11.60.2:45407): tablet 739fddf63e6645f0aaf946c1b1c9920f (table test-table [id=c1cba100f9414cff84c7a34cdd8774c8]) successfully deleted
I20260811 03:14:40.206892 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:40.167757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39486 (local address 127.11.60.1:34941)
0811 03:14:40.167986 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:40.167996 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:40.177914 (+ 9918us) server_negotiation.cc:407] Connection header received
0811 03:14:40.178480 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:40.178486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:40.178578 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:40.178667 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:40.181803 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.182914 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.192350 (+ 9436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.193179 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.193311 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:40.199594 (+ 6283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:40.199607 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:40.199623 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:40.199668 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:40.203294 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:40.204195 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:40.204205 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:40.204209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:40.204411 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:40.204865 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:40.204871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:40.204875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:40.205398 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:40.205772 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:40.206182 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 03:14:40.206488 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:14:40.207748 27048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:40.300987 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 26919
I20260811 03:14:40.328274 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 27060
Aug 11 03:14:40 dist-test-slave-tm1g krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:14:40.346031 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:40.329183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.11.60.2:45407)
0811 03:14:40.329457 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:40.329464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:40.329486 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:14:40.329619 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:40.329631 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:40.329710 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:40.329841 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:40.331421 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.332291 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.333688 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.334182 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.334457 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:40.339437 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:40.339454 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:40.339475 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:40.339522 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:40.342924 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:40.343847 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:40.343854 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:40.343856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:40.344010 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:40.344462 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:40.344466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:40.344468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:40.344844 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:40.345109 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:40.345476 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 03:14:40.345678 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 03:14:40.346848 27198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:40.465075 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 27060
I20260811 03:14:40.486340 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 27211
Aug 11 03:14:40 dist-test-slave-tm1g krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:14:40.503320 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:40.487193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.11.60.3:37485)
0811 03:14:40.487385 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:40.487391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:40.487517 (+ 126us) server_negotiation.cc:407] Connection header received
0811 03:14:40.487737 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:40.487744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:40.487817 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:40.487945 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:40.489267 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.490009 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.491568 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:40.491975 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:40.492121 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:40.496861 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:40.496875 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:40.496893 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:40.496936 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:40.500077 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:40.500913 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:40.500921 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:40.500923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:40.501075 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:40.501504 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:40.501514 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:40.501516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:40.501899 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:40.502237 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:40.502663 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 03:14:40.502903 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 03:14:40.504009 27347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:40.596356 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 27211
I20260811 03:14:40.617229 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 26818
I20260811 03:14:40.618741 26887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:40.736548 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 26818
I20260811 03:14:40.772333 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:44637
--webserver_interface=127.11.60.62
--webserver_port=37809
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:44637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:41.140671 27409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:41.140998 27409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:41.141183 27409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:41.148831 27409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:14:41.148970 27409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:41.149062 27409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:41.149160 27409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:14:41.149250 27409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:14:41.159726 27409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:44637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:44637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=37809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:41.161217 27409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:41.162724 27409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:41.190843 27415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:41.190763 27416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:41.190781 27419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:41.191639 27409 server_base.cc:1034] running on GCE node
I20260811 03:14:41.192656 27409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:41.194576 27409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:41.195809 27409 hybrid_clock.cc:648] HybridClock initialized: now 1786418081195741 us; error 56 us; skew 500 ppm
I20260811 03:14:41.196336 27409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:41 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418081, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:41.200341 27409 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:14:41.202399 27409 webserver.cc:460] Webserver started at http://127.11.60.62:37809/ using document root <none> and password file <none>
I20260811 03:14:41.203311 27409 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:41.203503 27409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:41.210196 27409 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 03:14:41.213536 27427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:41.215242 27409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:14:41.215410 27409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "83b4d74dc47d4aa3afbd665961fa8c22"
format_stamp: "Formatted at 2026-08-11 03:14:37 on dist-test-slave-tm1g"
server_key: "4b43dd7811c224536e0041cdc5f51239"
server_key_iv: "3596fdd301bcd483c2243b94ad1730d9"
server_key_version: "encryptionkey@0"
I20260811 03:14:41.216552 27409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:41.245738 27409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:41.250497 27409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:41.250763 27409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:41.271560 27409 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:44637
I20260811 03:14:41.271584 27482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:44637 every 8 connection(s)
I20260811 03:14:41.273416 27409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:14:41.275674 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 27409
I20260811 03:14:41.276517 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:34941
--local_ip_for_outbound_sockets=127.11.60.1
--tserver_master_addrs=127.11.60.62:44637
--webserver_port=46627
--webserver_interface=127.11.60.1
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:14:41.281268 27483 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:14:41.284873 27483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Bootstrap starting.
I20260811 03:14:41.306641 27483 log.cc:824] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:41.320228 27483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:14:41.320881 27483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Bootstrap complete.
I20260811 03:14:41.330809 27483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:41.331758 27483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b4d74dc47d4aa3afbd665961fa8c22, State: Initialized, Role: FOLLOWER
I20260811 03:14:41.332387 27483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:41.332650 27483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:14:41.332792 27483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:14:41.332994 27483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 1 FOLLOWER]: Advancing to term 2
I20260811 03:14:41.337131 27483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:41.337728 27483 leader_election.cc:302] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83b4d74dc47d4aa3afbd665961fa8c22; no voters:
I20260811 03:14:41.338447 27483 leader_election.cc:288] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 03:14:41.338611 27487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 03:14:41.339993 27487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 2 LEADER]: Becoming Leader. State: Replica: 83b4d74dc47d4aa3afbd665961fa8c22, State: Running, Role: LEADER
I20260811 03:14:41.340607 27487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:41.341099 27483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:14:41.343212 27489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83b4d74dc47d4aa3afbd665961fa8c22. Latest consensus state: current_term: 2 leader_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } } }
I20260811 03:14:41.343637 27489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:41.344208 27488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } } }
I20260811 03:14:41.344559 27488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:41.348913 27493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:14:41.357379 27493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:14:41.359058 27493 catalog_manager.cc:1259] Loaded cluster ID: fc8d61ceca364394897d72f83ad16995
I20260811 03:14:41.359164 27493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:14:41.362555 27493 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 03:14:41.380640 27493 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 03:14:41.394994 27493 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:14:41.396888 27493 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Loaded TSK: 0
I20260811 03:14:41.398483 27493 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 03:14:41.657228 27485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:41.657609 27485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:41.657755 27485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:41.665697 27485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:41.665827 27485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:41.666010 27485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:14:41.676681 27485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:34941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=46627
--tserver_master_addrs=127.11.60.62:44637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:41.678118 27485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:41.679598 27485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:41.694095 27512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:41.694350 27513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:41.694870 27515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:41.695557 27485 server_base.cc:1034] running on GCE node
I20260811 03:14:41.696374 27485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:41.697794 27485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:41.699081 27485 hybrid_clock.cc:648] HybridClock initialized: now 1786418081699044 us; error 58 us; skew 500 ppm
I20260811 03:14:41.699509 27485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:41 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418081, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:41.703970 27485 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:14:41.706147 27485 webserver.cc:460] Webserver started at http://127.11.60.1:46627/ using document root <none> and password file <none>
I20260811 03:14:41.707227 27485 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:41.707397 27485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:41.714828 27485 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 03:14:41.718510 27523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:41.720366 27485 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 03:14:41.720582 27485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "9baa0849eddf431098946b6e1e31a1c3"
format_stamp: "Formatted at 2026-08-11 03:14:38 on dist-test-slave-tm1g"
server_key: "a7a840d45be373d4cc59ffd9c54d9da3"
server_key_iv: "4c4da85a6cdb7538ef518ac1a539d5ba"
server_key_version: "encryptionkey@0"
I20260811 03:14:41.722015 27485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:41.746084 27485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:41.752226 27485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:41.752553 27485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:41.754029 27485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:41.755908 27485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:41.756069 27485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 03:14:41.756203 27485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:41.756309 27485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:41.794678 27485 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:34941
I20260811 03:14:41.794777 27649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:34941 every 8 connection(s)
I20260811 03:14:41.796604 27485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:14:41.797982 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 27485
I20260811 03:14:41.798884 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:45407
--local_ip_for_outbound_sockets=127.11.60.2
--tserver_master_addrs=127.11.60.62:44637
--webserver_port=36011
--webserver_interface=127.11.60.2
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:14:41 dist-test-slave-tm1g krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418081, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:41.827745 27653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:41.800561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:35603)
0811 03:14:41.803488 (+ 2927us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:41.803535 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:41.805401 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:41.807362 (+ 1961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:41.807375 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:41.808086 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:41.808810 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:41.808835 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:41.810666 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:41.810671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:41.811962 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:41.811976 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:41.813598 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:41.813604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:41.813839 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:41.814721 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:41.814752 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:41.817384 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:41.822849 (+ 5465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:41.822860 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:41.822874 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:41.823385 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:41.823976 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:41.823980 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:41.823982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:41.824175 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:41.825324 (+ 1149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:41.825333 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:41.825703 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 03:14:41.825966 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 03:14:41.826490 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2637,"thread_start_us":2444,"threads_started":1}
I20260811 03:14:41.828246 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:41.800209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:35603 (local address 127.11.60.62:44637)
0811 03:14:41.802957 (+ 2748us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:41.802972 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:41.803703 (+ 731us) server_negotiation.cc:407] Connection header received
0811 03:14:41.806190 (+ 2487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:41.806207 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:41.806765 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:41.807196 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:41.809078 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:41.810470 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:41.812133 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:41.813305 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:41.813530 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:41.817764 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:41.817778 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:41.817805 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:41.817874 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:41.822620 (+ 4746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:41.823562 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:41.823574 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:41.823584 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:41.823831 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:41.824298 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:41.824306 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:41.824312 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:41.825118 (+ 806us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:41.825363 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:41.826276 (+ 913us) server_negotiation.cc:299] Negotiation successful
0811 03:14:41.826833 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2471,"thread_start_us":2321,"threads_started":1}
I20260811 03:14:41.831120 27650 heartbeater.cc:347] Connected to a master server at 127.11.60.62:44637
I20260811 03:14:41.831444 27650 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:41.832228 27650 heartbeater.cc:511] Master 127.11.60.62:44637 requested a full tablet report, sending...
I20260811 03:14:41.834501 27445 ts_manager.cc:194] Registered new tserver with Master: 9baa0849eddf431098946b6e1e31a1c3 (127.11.60.1:34941)
I20260811 03:14:41.836040 27445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:35603
W20260811 03:14:42.173110 27654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:42.173465 27654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:42.173637 27654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:42.181221 27654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:42.181422 27654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:42.181613 27654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:14:42.192339 27654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:45407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=36011
--tserver_master_addrs=127.11.60.62:44637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:42.193933 27654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:42.195442 27654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:42.209745 27659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:42.209756 27660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:42.209942 27662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:42.210532 27654 server_base.cc:1034] running on GCE node
I20260811 03:14:42.211303 27654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:42.212334 27654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:42.213550 27654 hybrid_clock.cc:648] HybridClock initialized: now 1786418082213503 us; error 33 us; skew 500 ppm
I20260811 03:14:42.213927 27654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:42 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418082, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:42.217314 27654 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:14:42.219128 27654 webserver.cc:460] Webserver started at http://127.11.60.2:36011/ using document root <none> and password file <none>
I20260811 03:14:42.220005 27654 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:42.220155 27654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:42.226128 27654 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 03:14:42.229218 27669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:42.230821 27654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:14:42.231022 27654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "f3560e8dbb3b45b786b624046bab9a7f"
format_stamp: "Formatted at 2026-08-11 03:14:38 on dist-test-slave-tm1g"
server_key: "cb8d970429a687a7f069a634a74d60bf"
server_key_iv: "06b51eb00e34653d9e0e2844c458a5c0"
server_key_version: "encryptionkey@0"
I20260811 03:14:42.232067 27654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:42.287521 27654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:42.292271 27654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:42.292588 27654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:42.293708 27654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:42.295408 27654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:42.295552 27654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:42.295691 27654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:42.295785 27654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:42.333078 27654 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:45407
I20260811 03:14:42.333173 27795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:45407 every 8 connection(s)
I20260811 03:14:42.334761 27654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:14:42.336462 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 27654
I20260811 03:14:42.337249 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:37485
--local_ip_for_outbound_sockets=127.11.60.3
--tserver_master_addrs=127.11.60.62:44637
--webserver_port=42529
--webserver_interface=127.11.60.3
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:14:42 dist-test-slave-tm1g krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418082, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:42.362102 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.337929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:43463 (local address 127.11.60.62:44637)
0811 03:14:42.342764 (+ 4835us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:42.342768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:42.342790 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:14:42.343045 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:42.343049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:42.343115 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:42.343213 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:42.345515 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.346260 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.348979 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.349735 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.349913 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.354249 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:42.354259 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:42.354267 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:42.354346 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:42.358353 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:42.359197 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:42.359206 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:42.359213 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:42.359381 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:42.359856 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:42.359864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:42.359869 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:42.360308 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:42.360483 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:42.361272 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 03:14:42.361579 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4730,"thread_start_us":4576,"threads_started":1}
I20260811 03:14:42.362828 27800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.338688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:43463)
0811 03:14:42.339787 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:42.339840 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:42.342722 (+ 2882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:42.343538 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:42.343553 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:42.344386 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:42.345268 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:42.345298 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.346485 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.346493 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:42.348745 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:42.348764 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.349996 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.350002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:42.350214 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.351091 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:42.351128 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:42.353987 (+ 2859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:42.358527 (+ 4540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:42.358537 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:42.358551 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:42.359057 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:42.359507 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:42.359510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:42.359513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:42.359724 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:42.360452 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:42.360458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:42.360810 (+ 352us) client_negotiation.cc:769] Sending connection context
0811 03:14:42.361088 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 03:14:42.361549 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":396,"threads_started":1}
I20260811 03:14:42.364670 27796 heartbeater.cc:347] Connected to a master server at 127.11.60.62:44637
I20260811 03:14:42.365020 27796 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:42.365798 27796 heartbeater.cc:511] Master 127.11.60.62:44637 requested a full tablet report, sending...
I20260811 03:14:42.367519 27445 ts_manager.cc:194] Registered new tserver with Master: f3560e8dbb3b45b786b624046bab9a7f (127.11.60.2:45407)
I20260811 03:14:42.368911 27445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:43463
W20260811 03:14:42.699647 27801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:42.700062 27801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:42.700186 27801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:42.707638 27801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:42.707752 27801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:42.707870 27801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:14:42.717793 27801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:37485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=42529
--tserver_master_addrs=127.11.60.62:44637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:42.719131 27801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:42.720467 27801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:42.734486 27806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:42.734573 27807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:42.734963 27809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:42.735235 27801 server_base.cc:1034] running on GCE node
I20260811 03:14:42.736029 27801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:42.737089 27801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:42.738353 27801 hybrid_clock.cc:648] HybridClock initialized: now 1786418082738248 us; error 53 us; skew 500 ppm
I20260811 03:14:42.738679 27801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:42 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418082, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:42.742166 27801 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:14:42.743666 27801 webserver.cc:460] Webserver started at http://127.11.60.3:42529/ using document root <none> and password file <none>
I20260811 03:14:42.744366 27801 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:42.744520 27801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:42.749991 27801 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:14:42.752977 27816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:42.754431 27801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:14:42.754616 27801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "4d26616a9f79423b9d389bb0f55f6245"
format_stamp: "Formatted at 2026-08-11 03:14:39 on dist-test-slave-tm1g"
server_key: "47b1b2f43483da40e8f31db0f4e80948"
server_key_iv: "55192ff832bbcabc46c76e880725e9cd"
server_key_version: "encryptionkey@0"
I20260811 03:14:42.755638 27801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:42.769134 27801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:42.773707 27801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:42.774008 27801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:42.775115 27801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:42.776495 27801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:42.776640 27801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:42.776777 27801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:42.776870 27801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:42.813428 27801 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:37485
I20260811 03:14:42.813485 27943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:37485 every 8 connection(s)
I20260811 03:14:42.815068 27801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:14:42.823786 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 27801
Aug 11 03:14:42 dist-test-slave-tm1g krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418082, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:14:42.838677 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.817814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:52551 (local address 127.11.60.62:44637)
0811 03:14:42.818050 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:42.818060 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:42.819477 (+ 1417us) server_negotiation.cc:407] Connection header received
0811 03:14:42.821514 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:42.821518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:42.821597 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:42.821697 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:42.823790 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.824658 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.826896 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.827437 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.827586 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.831872 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:42.831877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:42.831880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:42.831913 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:42.834868 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:42.835734 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:42.835738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:42.835740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:42.835936 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:42.836499 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:42.836520 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:42.836526 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:42.836974 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:42.837159 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:42.838016 (+ 857us) server_negotiation.cc:299] Negotiation successful
0811 03:14:42.838266 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 03:14:42.839141 27650 heartbeater.cc:503] Master 127.11.60.62:44637 was elected leader, sending a full tablet report...
I20260811 03:14:42.839594 27947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.818292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:52551)
0811 03:14:42.819170 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:42.819218 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:42.821203 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:42.822002 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:42.822012 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:42.822692 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:14:42.823567 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:42.823600 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.824856 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.824860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:42.826749 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:42.826762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.827677 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.827684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:42.827921 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.828806 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:14:42.828844 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:14:42.831624 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:14:42.835016 (+ 3392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:42.835026 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:42.835044 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:42.835596 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:42.836048 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:14:42.836053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:14:42.836057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:14:42.836330 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:14:42.837095 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:14:42.837101 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:14:42.837520 (+ 419us) client_negotiation.cc:769] Sending connection context
0811 03:14:42.837826 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 03:14:42.838249 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":342,"threads_started":1}
I20260811 03:14:42.841225 27944 heartbeater.cc:347] Connected to a master server at 127.11.60.62:44637
I20260811 03:14:42.841540 27944 heartbeater.cc:464] Registering TS with master...
I20260811 03:14:42.842399 27944 heartbeater.cc:511] Master 127.11.60.62:44637 requested a full tablet report, sending...
I20260811 03:14:42.844162 27445 ts_manager.cc:194] Registered new tserver with Master: 4d26616a9f79423b9d389bb0f55f6245 (127.11.60.3:37485)
I20260811 03:14:42.844448 27948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.824894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56144 (local address 127.11.60.62:44637)
0811 03:14:42.825478 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:42.825485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:42.825887 (+ 402us) server_negotiation.cc:407] Connection header received
0811 03:14:42.826041 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:42.826048 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:42.826142 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:42.826250 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:42.828053 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.829186 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.831367 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.831881 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.832041 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.835557 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:42.835562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:42.835566 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:42.835600 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:42.839905 (+ 4305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:42.840801 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:42.840810 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:42.840813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:42.840964 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:42.842711 (+ 1747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:42.842718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:42.842722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:42.843120 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:42.843367 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:42.843755 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 03:14:42.844006 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":285,"threads_started":1}
I20260811 03:14:42.845077 27445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:52551
I20260811 03:14:42.847791 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:14:42.854795 27948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.848767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56158 (local address 127.11.60.62:44637)
0811 03:14:42.848946 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:42.848950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:42.849411 (+ 461us) server_negotiation.cc:407] Connection header received
0811 03:14:42.849631 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:42.849634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:42.849763 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:42.849881 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:14:42.850260 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.851092 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.852573 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.853037 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.853140 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.853630 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:14:42.854162 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:14:42.854273 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:42.854522 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 03:14:42.854596 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 03:14:42.855719 27445 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:56158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:14:42.863641 27445 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:14:42.902844 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.878634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39496 (local address 127.11.60.1:34941)
0811 03:14:42.892629 (+ 13995us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:42.892643 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:42.893314 (+ 671us) server_negotiation.cc:407] Connection header received
0811 03:14:42.893655 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:42.893664 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:42.893977 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:42.894192 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:42.895198 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.897253 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.899898 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.901554 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.901711 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.901975 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:42.902266 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 03:14:42.902477 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13799,"thread_start_us":13639,"threads_started":1}
I20260811 03:14:42.902936 27957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.877824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34941 (local address 127.0.0.1:39496)
0811 03:14:42.893178 (+ 15354us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:42.893200 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:42.893435 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:42.894433 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:42.894440 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:42.894477 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:42.894944 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:42.894965 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.897425 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.897431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:42.899717 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:42.899730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.901767 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.901774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:42.901985 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.902059 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 03:14:42.902207 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 03:14:42.902472 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15220,"mutex_wait_us":5,"thread_start_us":7270,"threads_started":1}
I20260811 03:14:42.903467 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.875864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:45407 (local address 127.0.0.1:34544)
0811 03:14:42.885700 (+ 9836us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:42.885774 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:42.886055 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:42.896088 (+ 10033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:42.896097 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:42.896127 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:42.896531 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:42.896545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.898487 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.898492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:42.900737 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:42.900753 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.902680 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.902686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:42.902895 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.902938 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 03:14:42.903009 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 03:14:42.903150 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9431,"thread_start_us":15865,"threads_started":1}
I20260811 03:14:42.904176 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.876096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:37485 (local address 127.0.0.1:34322)
0811 03:14:42.891421 (+ 15325us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:42.891456 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:42.891630 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:42.895696 (+ 4066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:42.895700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:42.895726 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:42.896155 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:42.896173 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.898043 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.898049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:42.900616 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:42.900635 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.902663 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.902672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:42.903516 (+ 844us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.903582 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 03:14:42.903791 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 03:14:42.903908 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15133,"mutex_wait_us":640,"thread_start_us":16758,"threads_started":1}
I20260811 03:14:42.904603 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.876249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34544 (local address 127.11.60.2:45407)
0811 03:14:42.895436 (+ 19187us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:42.895442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:42.895465 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:14:42.895551 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:42.895556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:42.895797 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:42.896080 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:42.896729 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.898339 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.900921 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.902415 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.903794 (+ 1379us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.903985 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:42.904053 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 03:14:42.904219 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19035,"thread_start_us":18884,"threads_started":1}
I20260811 03:14:42.905118 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:42.876450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.11.60.3:37485)
0811 03:14:42.895046 (+ 18596us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:42.895057 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:42.895100 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:14:42.895199 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:42.895206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:42.895462 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:42.895682 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:42.896385 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.897883 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.900831 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:42.902365 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:42.902911 (+ 546us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:42.903173 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:42.904609 (+ 1436us) server_negotiation.cc:299] Negotiation successful
0811 03:14:42.904802 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18378,"thread_start_us":18204,"threads_started":1}
I20260811 03:14:42.907720 27577 tablet_service.cc:1467] Processing CreateTablet for tablet 1c0e6eac3da74810b98e79a7f9285889 (DEFAULT_TABLE table=test-table [id=aa55006c49ad4442ab727207b9adeddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:14:42.908304 27721 tablet_service.cc:1467] Processing CreateTablet for tablet 1c0e6eac3da74810b98e79a7f9285889 (DEFAULT_TABLE table=test-table [id=aa55006c49ad4442ab727207b9adeddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:14:42.909138 27577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c0e6eac3da74810b98e79a7f9285889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:42.909646 27721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c0e6eac3da74810b98e79a7f9285889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:42.909307 27870 tablet_service.cc:1467] Processing CreateTablet for tablet 1c0e6eac3da74810b98e79a7f9285889 (DEFAULT_TABLE table=test-table [id=aa55006c49ad4442ab727207b9adeddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:14:42.910691 27870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c0e6eac3da74810b98e79a7f9285889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:14:42.922763 27963 tablet_bootstrap.cc:492] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: Bootstrap starting.
I20260811 03:14:42.925634 27964 tablet_bootstrap.cc:492] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: Bootstrap starting.
I20260811 03:14:42.926723 27963 tablet_bootstrap.cc:654] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:42.927410 27965 tablet_bootstrap.cc:492] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: Bootstrap starting.
I20260811 03:14:42.929445 27963 log.cc:824] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:42.929845 27964 tablet_bootstrap.cc:654] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:42.931052 27965 tablet_bootstrap.cc:654] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: Neither blocks nor log segments found. Creating new log.
I20260811 03:14:42.931237 27964 log.cc:824] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:42.932322 27965 log.cc:824] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:42.933111 27963 tablet_bootstrap.cc:492] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: No bootstrap required, opened a new log
I20260811 03:14:42.933447 27963 ts_tablet_manager.cc:1397] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260811 03:14:42.934329 27964 tablet_bootstrap.cc:492] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: No bootstrap required, opened a new log
I20260811 03:14:42.934626 27964 ts_tablet_manager.cc:1397] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260811 03:14:42.936731 27965 tablet_bootstrap.cc:492] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: No bootstrap required, opened a new log
I20260811 03:14:42.937151 27965 ts_tablet_manager.cc:1397] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.008s
I20260811 03:14:42.945643 27963 raft_consensus.cc:348] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } }
I20260811 03:14:42.946089 27963 raft_consensus.cc:374] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:42.946202 27963 raft_consensus.cc:729] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9baa0849eddf431098946b6e1e31a1c3, State: Initialized, Role: FOLLOWER
I20260811 03:14:42.946566 27964 raft_consensus.cc:348] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } }
I20260811 03:14:42.947041 27964 raft_consensus.cc:374] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:42.947162 27964 raft_consensus.cc:729] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3560e8dbb3b45b786b624046bab9a7f, State: Initialized, Role: FOLLOWER
I20260811 03:14:42.946971 27963 consensus_queue.cc:260] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } }
I20260811 03:14:42.947892 27964 consensus_queue.cc:260] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } }
I20260811 03:14:42.949143 27963 ts_tablet_manager.cc:1428] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: Time spent starting tablet: real 0.016s user 0.016s sys 0.001s
I20260811 03:14:42.952859 27965 raft_consensus.cc:348] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } }
I20260811 03:14:42.953480 27965 raft_consensus.cc:374] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:14:42.953608 27965 raft_consensus.cc:729] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d26616a9f79423b9d389bb0f55f6245, State: Initialized, Role: FOLLOWER
I20260811 03:14:42.954465 27965 consensus_queue.cc:260] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } }
I20260811 03:14:42.956770 27796 heartbeater.cc:503] Master 127.11.60.62:44637 was elected leader, sending a full tablet report...
I20260811 03:14:42.957427 27944 heartbeater.cc:503] Master 127.11.60.62:44637 was elected leader, sending a full tablet report...
I20260811 03:14:42.958520 27964 ts_tablet_manager.cc:1428] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: Time spent starting tablet: real 0.024s user 0.011s sys 0.010s
I20260811 03:14:42.959816 27965 ts_tablet_manager.cc:1428] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: Time spent starting tablet: real 0.020s user 0.015s sys 0.003s
W20260811 03:14:43.049465 27651 tablet.cc:2369] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:14:43.060988 27971 raft_consensus.cc:482] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:14:43.061318 27971 raft_consensus.cc:504] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } }
I20260811 03:14:43.063215 27971 leader_election.cc:288] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3560e8dbb3b45b786b624046bab9a7f (127.11.60.2:45407), 9baa0849eddf431098946b6e1e31a1c3 (127.11.60.1:34941)
W20260811 03:14:43.066821 27945 tablet.cc:2369] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:14:43.070117 27947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:43.063412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:45407 (local address 127.11.60.3:33133)
0811 03:14:43.063705 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:43.063727 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:43.063878 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:43.064331 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:43.064338 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:43.064378 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:43.064623 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:43.064639 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.066089 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.066093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:43.068099 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:43.068115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.069555 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.069560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:43.069698 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:43.069722 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:14:43.069804 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 03:14:43.069898 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 03:14:43.070294 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:43.063623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:33133 (local address 127.11.60.2:45407)
0811 03:14:43.063856 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:43.063862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:43.063887 (+ 25us) server_negotiation.cc:407] Connection header received
0811 03:14:43.064050 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:43.064058 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:43.064156 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:43.064286 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:43.064850 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.065929 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.068292 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.069391 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.069535 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:43.069646 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:43.069859 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 03:14:43.070007 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 03:14:43.071064 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:43.063968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:52483 (local address 127.11.60.1:34941)
0811 03:14:43.064155 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:43.064159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:43.064797 (+ 638us) server_negotiation.cc:407] Connection header received
0811 03:14:43.064915 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:43.064922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:43.065000 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:43.065102 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:14:43.065558 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.066581 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.068821 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.070056 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.070192 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:43.070676 (+ 484us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:43.070732 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 03:14:43.070888 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 03:14:43.071113 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:43.063824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34941 (local address 127.11.60.3:52483)
0811 03:14:43.064661 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:43.064679 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:14:43.064799 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:14:43.065117 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:14:43.065120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:14:43.065139 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:14:43.065411 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:43.065418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.066749 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.066754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:43.068665 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:14:43.068677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.070277 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.070283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:14:43.070505 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:43.070529 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:14:43.070598 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 03:14:43.070690 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":1,"thread_start_us":494,"threads_started":1}
I20260811 03:14:43.071390 27746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0e6eac3da74810b98e79a7f9285889" candidate_uuid: "4d26616a9f79423b9d389bb0f55f6245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3560e8dbb3b45b786b624046bab9a7f" is_pre_election: true
I20260811 03:14:43.072000 27746 raft_consensus.cc:2454] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d26616a9f79423b9d389bb0f55f6245 in term 0.
I20260811 03:14:43.072247 27597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0e6eac3da74810b98e79a7f9285889" candidate_uuid: "4d26616a9f79423b9d389bb0f55f6245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9baa0849eddf431098946b6e1e31a1c3" is_pre_election: true
I20260811 03:14:43.072746 27818 leader_election.cc:302] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d26616a9f79423b9d389bb0f55f6245, f3560e8dbb3b45b786b624046bab9a7f; no voters:
I20260811 03:14:43.072899 27597 raft_consensus.cc:2454] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d26616a9f79423b9d389bb0f55f6245 in term 0.
I20260811 03:14:43.073222 27971 raft_consensus.cc:2781] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:14:43.073396 27971 raft_consensus.cc:482] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:14:43.073529 27971 raft_consensus.cc:3037] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:43.075914 27971 raft_consensus.cc:504] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } }
I20260811 03:14:43.076526 27971 leader_election.cc:288] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [CANDIDATE]: Term 1 election: Requested vote from peers f3560e8dbb3b45b786b624046bab9a7f (127.11.60.2:45407), 9baa0849eddf431098946b6e1e31a1c3 (127.11.60.1:34941)
I20260811 03:14:43.077019 27746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0e6eac3da74810b98e79a7f9285889" candidate_uuid: "4d26616a9f79423b9d389bb0f55f6245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3560e8dbb3b45b786b624046bab9a7f"
I20260811 03:14:43.077032 27597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0e6eac3da74810b98e79a7f9285889" candidate_uuid: "4d26616a9f79423b9d389bb0f55f6245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9baa0849eddf431098946b6e1e31a1c3"
I20260811 03:14:43.077255 27597 raft_consensus.cc:3037] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:43.077250 27746 raft_consensus.cc:3037] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:14:43.079600 27597 raft_consensus.cc:2454] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d26616a9f79423b9d389bb0f55f6245 in term 1.
I20260811 03:14:43.079723 27746 raft_consensus.cc:2454] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d26616a9f79423b9d389bb0f55f6245 in term 1.
I20260811 03:14:43.080094 27820 leader_election.cc:302] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d26616a9f79423b9d389bb0f55f6245, 9baa0849eddf431098946b6e1e31a1c3; no voters:
I20260811 03:14:43.080451 27971 raft_consensus.cc:2781] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:14:43.080941 27971 raft_consensus.cc:686] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 1 LEADER]: Becoming Leader. State: Replica: 4d26616a9f79423b9d389bb0f55f6245, State: Running, Role: LEADER
I20260811 03:14:43.081609 27971 consensus_queue.cc:237] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } } peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } }
W20260811 03:14:43.087101 27797 tablet.cc:2369] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:14:43.086500 27444 catalog_manager.cc:5495] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d26616a9f79423b9d389bb0f55f6245 (127.11.60.3). New cstate: current_term: 1 leader_uuid: "4d26616a9f79423b9d389bb0f55f6245" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d26616a9f79423b9d389bb0f55f6245" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 } health_report { overall_health: UNKNOWN } } }
I20260811 03:14:43.105736 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:43.099714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34338 (local address 127.11.60.3:37485)
0811 03:14:43.099953 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:43.099957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:43.100031 (+ 74us) server_negotiation.cc:407] Connection header received
0811 03:14:43.100217 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:43.100221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:43.100295 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:43.100416 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:14:43.100868 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.101623 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.103514 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.103914 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.104011 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:43.104484 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:14:43.105134 (+ 650us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:14:43.105253 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:43.105374 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 03:14:43.105445 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 03:14:43.114714 27597 raft_consensus.cc:1261] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d26616a9f79423b9d389bb0f55f6245: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:14:43.114713 27746 raft_consensus.cc:1261] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f [term 1 FOLLOWER]: Refusing update from remote peer 4d26616a9f79423b9d389bb0f55f6245: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:14:43.115785 27973 consensus_queue.cc:1035] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9baa0849eddf431098946b6e1e31a1c3" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:14:43.116146 27971 consensus_queue.cc:1035] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3560e8dbb3b45b786b624046bab9a7f" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 45407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:14:43.127740 27977 mvcc.cc:205] Tried to move back new op lower bound from 7317168468427530240 to 7317168468306755584. Current Snapshot: MvccSnapshot[applied={T|T < 7317168468427530240}]
I20260811 03:14:43.150557 27444 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:56158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:14:43.150857 27444 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:56158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:14:43.154629 27444 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Sending DeleteTablet for 3 replicas of tablet 1c0e6eac3da74810b98e79a7f9285889
I20260811 03:14:43.155921 27721 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c0e6eac3da74810b98e79a7f9285889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:14:43 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:34544
I20260811 03:14:43.156064 27577 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c0e6eac3da74810b98e79a7f9285889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:14:43 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:39496
I20260811 03:14:43.156236 27870 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c0e6eac3da74810b98e79a7f9285889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:14:43 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:34322
I20260811 03:14:43.157685 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 27485
I20260811 03:14:43.167070 27989 tablet_replica.cc:330] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: stopping tablet replica
I20260811 03:14:43.167703 27989 raft_consensus.cc:2229] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:14:43.168390 27989 raft_consensus.cc:2258] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:14:43.170969 27989 ts_tablet_manager.cc:1905] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:14:43.177330 27990 tablet_replica.cc:330] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: stopping tablet replica
I20260811 03:14:43.177830 27990 raft_consensus.cc:2229] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:14:43.179092 27989 ts_tablet_manager.cc:1918] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:14:43.179325 27989 log.cc:1196] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/1c0e6eac3da74810b98e79a7f9285889
I20260811 03:14:43.180047 27989 ts_tablet_manager.cc:1939] T 1c0e6eac3da74810b98e79a7f9285889 P 9baa0849eddf431098946b6e1e31a1c3: Deleting consensus metadata
I20260811 03:14:43.180804 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:43.158647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39502 (local address 127.11.60.1:34941)
0811 03:14:43.158826 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:43.158833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:43.159062 (+ 229us) server_negotiation.cc:407] Connection header received
0811 03:14:43.159270 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:43.159277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:43.159407 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:43.159519 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:43.161824 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.163515 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.166112 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.167190 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.167321 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:43.171701 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:43.171721 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:43.171746 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:43.171804 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:43.177047 (+ 5243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:43.177967 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:43.177979 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:43.177986 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:43.178247 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:43.178707 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:43.178715 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:43.178720 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:43.179373 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:43.179741 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:43.180093 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 03:14:43.180339 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 03:14:43.181635 27645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:43.182353 27990 raft_consensus.cc:2258] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:14:43.183966 27988 tablet_replica.cc:330] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: stopping tablet replica
I20260811 03:14:43.184449 27988 raft_consensus.cc:2229] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:14:43.184995 27988 raft_consensus.cc:2258] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:14:43.187487 27990 ts_tablet_manager.cc:1905] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:14:43.191022 27988 ts_tablet_manager.cc:1905] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:14:43.195183 27990 ts_tablet_manager.cc:1918] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:14:43.195384 27990 log.cc:1196] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/1c0e6eac3da74810b98e79a7f9285889
I20260811 03:14:43.196023 27990 ts_tablet_manager.cc:1939] T 1c0e6eac3da74810b98e79a7f9285889 P 4d26616a9f79423b9d389bb0f55f6245: Deleting consensus metadata
I20260811 03:14:43.198017 27431 catalog_manager.cc:4841] TS 4d26616a9f79423b9d389bb0f55f6245 (127.11.60.3:37485): tablet 1c0e6eac3da74810b98e79a7f9285889 (table test-table [id=aa55006c49ad4442ab727207b9adeddd]) successfully deleted
I20260811 03:14:43.199383 27988 ts_tablet_manager.cc:1918] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:14:43.199574 27988 log.cc:1196] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/1c0e6eac3da74810b98e79a7f9285889
I20260811 03:14:43.200160 27988 ts_tablet_manager.cc:1939] T 1c0e6eac3da74810b98e79a7f9285889 P f3560e8dbb3b45b786b624046bab9a7f: Deleting consensus metadata
I20260811 03:14:43.202265 27428 catalog_manager.cc:4841] TS f3560e8dbb3b45b786b624046bab9a7f (127.11.60.2:45407): tablet 1c0e6eac3da74810b98e79a7f9285889 (table test-table [id=aa55006c49ad4442ab727207b9adeddd]) successfully deleted
I20260811 03:14:43.284050 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 27485
I20260811 03:14:43.284276 27430 catalog_manager.cc:4841] TS 9baa0849eddf431098946b6e1e31a1c3 (127.11.60.1:34941): tablet 1c0e6eac3da74810b98e79a7f9285889 (table test-table [id=aa55006c49ad4442ab727207b9adeddd]) successfully deleted
I20260811 03:14:43.305747 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 27654
I20260811 03:14:43.322820 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:43.306708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34560 (local address 127.11.60.2:45407)
0811 03:14:43.306912 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:43.306917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:43.306960 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:14:43.307192 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:43.307196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:43.307276 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:43.307359 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:43.309273 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.310058 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.312071 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.312543 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.312668 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:43.316280 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:43.316301 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:43.316320 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:43.316360 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:43.319649 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:43.320571 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:43.320579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:43.320582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:43.320712 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:43.321139 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:43.321144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:43.321148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:43.321632 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:43.321863 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:43.322272 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 03:14:43.322508 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 03:14:43.323623 27791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:43.417212 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 27654
I20260811 03:14:43.438524 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 27801
I20260811 03:14:43.456203 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:43.439560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.11.60.3:37485)
0811 03:14:43.439809 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 03:14:43.439822 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:14:43.439845 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:14:43.439917 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:14:43.439923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:14:43.440013 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:14:43.440152 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:14:43.441622 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.443041 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.444390 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:14:43.445166 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:14:43.445331 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:14:43.448118 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:14:43.448132 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:14:43.448151 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:14:43.448198 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:14:43.452797 (+ 4599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:43.453607 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:43.453620 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:43.453628 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:43.453840 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:14:43.454263 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:14:43.454269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:14:43.454272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:14:43.454813 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:14:43.455159 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 03:14:43.455483 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 03:14:43.455726 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 03:14:43.457192 27939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:43.585440 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 27801
I20260811 03:14:43.609063 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 27409
I20260811 03:14:43.610380 27478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:14:43.729983 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 27409
I20260811 03:14:43.759438 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:44637
--webserver_interface=127.11.60.62
--webserver_port=37809
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:44637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:44.122834 27995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:44.123229 27995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:44.123412 27995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:44.131631 27995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:14:44.131785 27995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:44.131875 27995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:44.131964 27995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:14:44.132031 27995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:14:44.142408 27995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:44637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:44637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=37809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:44.143941 27995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:44.145442 27995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:44.158104 28005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:44.158231 28001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:44.158080 28003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:44.158878 27995 server_base.cc:1034] running on GCE node
I20260811 03:14:44.159752 27995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:44.161319 27995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:44.162561 27995 hybrid_clock.cc:648] HybridClock initialized: now 1786418084162505 us; error 78 us; skew 500 ppm
I20260811 03:14:44.162947 27995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:44 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418084, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:44.166201 27995 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:14:44.167883 27995 webserver.cc:460] Webserver started at http://127.11.60.62:37809/ using document root <none> and password file <none>
I20260811 03:14:44.168653 27995 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:44.168813 27995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:44.174850 27995 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 03:14:44.180398 28012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:44.181922 27995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:14:44.182157 27995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "83b4d74dc47d4aa3afbd665961fa8c22"
format_stamp: "Formatted at 2026-08-11 03:14:37 on dist-test-slave-tm1g"
server_key: "4b43dd7811c224536e0041cdc5f51239"
server_key_iv: "3596fdd301bcd483c2243b94ad1730d9"
server_key_version: "encryptionkey@0"
I20260811 03:14:44.183243 27995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:44.224442 27995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:44.229328 27995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:44.229662 27995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:44.251823 27995 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:44637
I20260811 03:14:44.251855 28070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:44637 every 8 connection(s)
I20260811 03:14:44.253592 27995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:14:44.260727 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 27995
I20260811 03:14:44.261936 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:34941
--local_ip_for_outbound_sockets=127.11.60.1
--tserver_master_addrs=127.11.60.62:44637
--webserver_port=46627
--webserver_interface=127.11.60.1
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:14:44.262085 28071 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:14:44.268283 28071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Bootstrap starting.
I20260811 03:14:44.295449 28071 log.cc:824] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Log is configured to *not* fsync() on all Append() calls
I20260811 03:14:44.317834 28071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:14:44.318625 28071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22: Bootstrap complete.
I20260811 03:14:44.330055 28071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:44.331382 28071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b4d74dc47d4aa3afbd665961fa8c22, State: Initialized, Role: FOLLOWER
I20260811 03:14:44.332192 28071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:44.332463 28071 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:14:44.332595 28071 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:14:44.332820 28071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 2 FOLLOWER]: Advancing to term 3
I20260811 03:14:44.337045 28071 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:44.337596 28071 leader_election.cc:302] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83b4d74dc47d4aa3afbd665961fa8c22; no voters:
I20260811 03:14:44.338385 28071 leader_election.cc:288] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 03:14:44.338546 28075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 03:14:44.339676 28075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [term 3 LEADER]: Becoming Leader. State: Replica: 83b4d74dc47d4aa3afbd665961fa8c22, State: Running, Role: LEADER
I20260811 03:14:44.340327 28075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } }
I20260811 03:14:44.340833 28071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:14:44.343123 28077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83b4d74dc47d4aa3afbd665961fa8c22. Latest consensus state: current_term: 3 leader_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } } }
I20260811 03:14:44.343524 28077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:44.344209 28076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b4d74dc47d4aa3afbd665961fa8c22" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 44637 } } }
I20260811 03:14:44.344496 28076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b4d74dc47d4aa3afbd665961fa8c22 [sys.catalog]: This master's current role is: LEADER
I20260811 03:14:44.349720 28081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:14:44.362366 28081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:14:44.364681 28081 catalog_manager.cc:1259] Loaded cluster ID: fc8d61ceca364394897d72f83ad16995
I20260811 03:14:44.364835 28081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 03:14:44.367978 28081 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f66f52849ad google::LogMessage::Fail() at ??:0
@ 0x7f66f528788a google::LogMessage::SendToLog() at ??:0
@ 0x7f66f52844de google::LogMessage::Flush() at ??:0
@ 0x7f66f5285a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f670acb7512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
I20260811 03:14:44.389873 28095 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f670acb4976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f670ad3c610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f670ad3c439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f66f7ed0345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f66f7ed391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f66f7ed3799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f66f7eaeeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f66fb1886db start_thread at ??:0
@ 0x7f66f411e61f clone at ??:0
W20260811 03:14:44.680924 28073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:44.681286 28073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:44.681484 28073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:44.689469 28073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:44.689647 28073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:44.689832 28073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:14:44.700330 28073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:34941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=46627
--tserver_master_addrs=127.11.60.62:44637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:44.701797 28073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:44.703272 28073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:44.717366 28116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:44.717624 28117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:44.717615 28119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:44.719100 28073 server_base.cc:1034] running on GCE node
I20260811 03:14:44.719797 28073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:44.720860 28073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:44.722131 28073 hybrid_clock.cc:648] HybridClock initialized: now 1786418084722079 us; error 49 us; skew 500 ppm
I20260811 03:14:44.722553 28073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:44 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418084, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:44.725919 28073 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:14:44.727447 28073 webserver.cc:460] Webserver started at http://127.11.60.1:46627/ using document root <none> and password file <none>
I20260811 03:14:44.728164 28073 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:44.728276 28073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:44.733800 28073 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 03:14:44.736651 28127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:44.738003 28073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:14:44.738155 28073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "9baa0849eddf431098946b6e1e31a1c3"
format_stamp: "Formatted at 2026-08-11 03:14:38 on dist-test-slave-tm1g"
server_key: "a7a840d45be373d4cc59ffd9c54d9da3"
server_key_iv: "4c4da85a6cdb7538ef518ac1a539d5ba"
server_key_version: "encryptionkey@0"
I20260811 03:14:44.739207 28073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:44.777711 28073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:44.782089 28073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:44.782416 28073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:44.783586 28073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:44.784991 28073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:44.785095 28073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:44.785174 28073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:44.785233 28073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:44.822700 28073 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:34941
I20260811 03:14:44.822824 28253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:34941 every 8 connection(s)
I20260811 03:14:44.824294 28073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:14:44.829238 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:44.827322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:38141)
0811 03:14:44.828117 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:44.828327 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":288,"threads_started":1}
W20260811 03:14:44.829977 28254 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:44637 (0 consecutive failures): Network error: Failed to ping master at 127.11.60.62:44637: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
I20260811 03:14:44.830888 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:44.830543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:39021)
0811 03:14:44.830708 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:44.830778 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 03:14:44.831789 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:44.831434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:34821)
0811 03:14:44.831607 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:44.831686 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 03:14:44.832041 28254 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 03:14:44.834731 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 28073
I20260811 03:14:44.835356 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:45407
--local_ip_for_outbound_sockets=127.11.60.2
--tserver_master_addrs=127.11.60.62:44637
--webserver_port=36011
--webserver_interface=127.11.60.2
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:14:45.201442 28257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:45.201774 28257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:45.201887 28257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:45.209400 28257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:45.209527 28257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:45.209647 28257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:14:45.220808 28257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:45407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=36011
--tserver_master_addrs=127.11.60.62:44637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:45.222244 28257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:45.223703 28257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:45.238127 28264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:45.238090 28267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:45.238031 28265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:45.238425 28257 server_base.cc:1034] running on GCE node
I20260811 03:14:45.239446 28257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:45.240401 28257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:45.241636 28257 hybrid_clock.cc:648] HybridClock initialized: now 1786418085241550 us; error 71 us; skew 500 ppm
I20260811 03:14:45.241950 28257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:45 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418085, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:45.245697 28257 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:14:45.247385 28257 webserver.cc:460] Webserver started at http://127.11.60.2:36011/ using document root <none> and password file <none>
I20260811 03:14:45.248215 28257 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:45.248389 28257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:45.254745 28257 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 03:14:45.257853 28277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:45.259500 28257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:14:45.259722 28257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "f3560e8dbb3b45b786b624046bab9a7f"
format_stamp: "Formatted at 2026-08-11 03:14:38 on dist-test-slave-tm1g"
server_key: "cb8d970429a687a7f069a634a74d60bf"
server_key_iv: "06b51eb00e34653d9e0e2844c458a5c0"
server_key_version: "encryptionkey@0"
I20260811 03:14:45.260833 28257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:45.289328 28257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:45.293802 28257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:45.294118 28257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:45.295353 28257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:45.297191 28257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:45.297283 28257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:45.297361 28257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:45.297416 28257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:45.332798 28257 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:45407
I20260811 03:14:45.332896 28408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:45407 every 8 connection(s)
I20260811 03:14:45.334457 28257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:14:45.335551 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 28257
I20260811 03:14:45.336308 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:37485
--local_ip_for_outbound_sockets=127.11.60.3
--tserver_master_addrs=127.11.60.62:44637
--webserver_port=42529
--webserver_interface=127.11.60.3
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:14:45.340415 28411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:45.338493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:55875)
0811 03:14:45.339305 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:45.339518 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":319,"threads_started":1}
W20260811 03:14:45.341254 28409 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:44637 (0 consecutive failures): Network error: Failed to ping master at 127.11.60.62:44637: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
I20260811 03:14:45.342255 28411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:45.341911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:38133)
0811 03:14:45.342076 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:45.342150 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 03:14:45.343909 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:45.342792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:51857)
0811 03:14:45.343402 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:45.343571 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":242,"threads_started":1}
W20260811 03:14:45.344198 28409 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 03:14:45.690030 28413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:14:45.690438 28413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:14:45.690608 28413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:14:45.698061 28413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:14:45.698230 28413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:14:45.698426 28413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:14:45.708724 28413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:37485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=42529
--tserver_master_addrs=127.11.60.62:44637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:14:45.710165 28413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:14:45.711848 28413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:14:45.726413 28418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:45.726503 28421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:14:45.726466 28419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:14:45.727507 28413 server_base.cc:1034] running on GCE node
I20260811 03:14:45.728185 28413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:14:45.729192 28413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:14:45.730448 28413 hybrid_clock.cc:648] HybridClock initialized: now 1786418085730389 us; error 43 us; skew 500 ppm
I20260811 03:14:45.730837 28413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:14:45 dist-test-slave-tm1g krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418085, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:14:45.734393 28413 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:14:45.735977 28413 webserver.cc:460] Webserver started at http://127.11.60.3:42529/ using document root <none> and password file <none>
I20260811 03:14:45.736739 28413 fs_manager.cc:359] Metadata directory not provided
I20260811 03:14:45.736896 28413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:14:45.743539 28413 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 03:14:45.747418 28431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:45.749373 28413 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 03:14:45.749565 28413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "4d26616a9f79423b9d389bb0f55f6245"
format_stamp: "Formatted at 2026-08-11 03:14:39 on dist-test-slave-tm1g"
server_key: "47b1b2f43483da40e8f31db0f4e80948"
server_key_iv: "55192ff832bbcabc46c76e880725e9cd"
server_key_version: "encryptionkey@0"
I20260811 03:14:45.750931 28413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:14:45.785427 28413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:14:45.790717 28413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:14:45.791020 28413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:14:45.792189 28413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:14:45.793704 28413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:14:45.793815 28413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:45.793911 28413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:14:45.793974 28413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:14:45.831853 28413 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:37485
I20260811 03:14:45.832018 28560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:37485 every 8 connection(s)
I20260811 03:14:45.833738 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:45.832797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:43683)
0811 03:14:45.833358 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:45.833489 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":263,"threads_started":1}
I20260811 03:14:45.833766 28413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:14:45.838250 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 28413
I20260811 03:14:45.839355 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:45.836929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:54547)
0811 03:14:45.837852 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:45.838151 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":431,"threads_started":1}
W20260811 03:14:45.840134 28562 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:44637 (0 consecutive failures): Network error: Failed to ping master at 127.11.60.62:44637: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
I20260811 03:14:45.841048 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:45.840679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:44809)
0811 03:14:45.840836 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:45.840916 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 03:14:45.841854 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:45.841559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:35355)
0811 03:14:45.841683 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:45.841754 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 03:14:45.842108 28562 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 03:14:46.345538 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:46.344785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:48307)
0811 03:14:46.345222 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:46.345321 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":219,"threads_started":1}
I20260811 03:14:46.835695 28568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:46.834911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:37477)
0811 03:14:46.835399 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:46.835520 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":241,"threads_started":1}
I20260811 03:14:46.843773 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:46.842752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:53205)
0811 03:14:46.843371 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:46.843513 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":252,"threads_started":1}
I20260811 03:14:47.347286 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:47.346565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:41447)
0811 03:14:47.347041 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:47.347121 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":241,"threads_started":1}
I20260811 03:14:47.837623 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:47.836672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:52645)
0811 03:14:47.837279 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:47.837370 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":390,"threads_started":1}
I20260811 03:14:47.845468 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:47.844717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:48471)
0811 03:14:47.845194 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:47.845272 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":251,"threads_started":1}
I20260811 03:14:48.348984 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:48.348250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:53945)
0811 03:14:48.348692 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:48.348792 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":231,"threads_started":1}
I20260811 03:14:48.839589 28574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:48.838725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:33421)
0811 03:14:48.839297 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:48.839388 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":316,"threads_started":1}
I20260811 03:14:48.847149 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:48.846427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:35797)
0811 03:14:48.846874 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:48.846958 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":249,"threads_started":1}
I20260811 03:14:49.350828 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:49.349997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:59391)
0811 03:14:49.350565 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:49.350664 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":236,"threads_started":1}
I20260811 03:14:49.841243 28577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:49.840504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:50929)
0811 03:14:49.840988 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:49.841073 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":260,"threads_started":1}
I20260811 03:14:49.848780 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:49.848007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:42549)
0811 03:14:49.848544 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:49.848619 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":296,"threads_started":1}
I20260811 03:14:50.352557 28579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:50.351811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:51885)
0811 03:14:50.352289 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:50.352376 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":227,"threads_started":1}
I20260811 03:14:50.843161 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:50.842372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:47389)
0811 03:14:50.842894 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:50.842995 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":294,"threads_started":1}
I20260811 03:14:50.850628 28582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:50.849781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:52215)
0811 03:14:50.850273 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:50.850457 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":238,"threads_started":1}
I20260811 03:14:51.354164 28583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:51.353431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:39111)
0811 03:14:51.353896 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:51.353993 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":236,"threads_started":1}
I20260811 03:14:51.845005 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:51.844166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:54629)
0811 03:14:51.844696 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:51.844785 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":246,"threads_started":1}
I20260811 03:14:51.852557 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:51.851656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:54205)
0811 03:14:51.852222 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:51.852342 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":248,"threads_started":1}
I20260811 03:14:52.355916 28587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:52.355095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:59673)
0811 03:14:52.355613 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:52.355694 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":192,"threads_started":1}
I20260811 03:14:52.846866 28588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:52.846004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:46469)
0811 03:14:52.846575 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:52.846665 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":324,"threads_started":1}
I20260811 03:14:52.854517 28589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:52.853506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:39441)
0811 03:14:52.854123 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:52.854219 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":312,"threads_started":1}
I20260811 03:14:53.357760 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:53.356964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:45487)
0811 03:14:53.357511 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:53.357590 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":306,"threads_started":1}
I20260811 03:14:53.848627 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:53.847793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:43377)
0811 03:14:53.848347 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:53.848440 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":315,"threads_started":1}
I20260811 03:14:53.856196 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:53.855499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:42899)
0811 03:14:53.855946 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:53.856026 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":233,"threads_started":1}
I20260811 03:14:54.359395 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:54.358666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:43777)
0811 03:14:54.359109 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:54.359199 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":218,"threads_started":1}
I20260811 03:14:54.850481 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:54.849699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:50993)
0811 03:14:54.850165 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:54.850251 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":248,"threads_started":1}
I20260811 03:14:54.857632 28597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:54.856971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:47219)
0811 03:14:54.857392 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:54.857466 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":201,"threads_started":1}
I20260811 03:14:55.361224 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:55.360353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:53461)
0811 03:14:55.360910 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:55.361011 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":301,"threads_started":1}
I20260811 03:14:55.852170 28599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:55.851432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:35887)
0811 03:14:55.851886 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:55.851990 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":238,"threads_started":1}
I20260811 03:14:55.859176 28600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:55.858503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:51707)
0811 03:14:55.858922 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:55.859016 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":219,"threads_started":1}
I20260811 03:14:56.363016 28601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:56.362240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:38539)
0811 03:14:56.362723 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:56.362822 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":255,"threads_started":1}
I20260811 03:14:56.854034 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:56.853216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:37957)
0811 03:14:56.853698 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:56.853820 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":250,"threads_started":1}
I20260811 03:14:56.860728 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:56.860005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:49769)
0811 03:14:56.860435 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:56.860540 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":215,"threads_started":1}
I20260811 03:14:57.364764 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:57.364040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:51075)
0811 03:14:57.364495 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:57.364583 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":262,"threads_started":1}
I20260811 03:14:57.855851 28605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:57.855062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:42825)
0811 03:14:57.855544 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:57.855639 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":229,"threads_started":1}
I20260811 03:14:57.862480 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:57.861686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:42351)
0811 03:14:57.862186 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:57.862261 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":290,"threads_started":1}
I20260811 03:14:58.366700 28607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:58.365823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:34957)
0811 03:14:58.366420 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:58.366505 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":260,"threads_started":1}
I20260811 03:14:58.857461 28608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:58.856726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:45983)
0811 03:14:58.857191 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:58.857277 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":254,"threads_started":1}
I20260811 03:14:58.864157 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:58.863293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:39541)
0811 03:14:58.863849 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:58.863945 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":323,"threads_started":1}
I20260811 03:14:59.368381 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:59.367591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:36587)
0811 03:14:59.368083 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:59.368169 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":255,"threads_started":1}
I20260811 03:14:59.859377 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:59.858586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:40451)
0811 03:14:59.859080 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:59.859180 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":272,"threads_started":1}
I20260811 03:14:59.865815 28612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:14:59.865017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:58253)
0811 03:14:59.865547 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 03:14:59.865629 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":295,"threads_started":1}
I20260811 03:15:00.370395 28614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:00.369521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.2:45389)
0811 03:15:00.370005 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:00.370129 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":254,"threads_started":1}
I20260811 03:15:00.842936 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 28073
I20260811 03:15:00.861519 28616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:00.860392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.1:48093)
0811 03:15:00.861062 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:00.861216 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":373,"threads_started":1}
I20260811 03:15:00.867239 28615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:00.844178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36456 (local address 127.11.60.1:34941)
0811 03:15:00.844936 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:00.844954 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:00.845000 (+ 46us) server_negotiation.cc:407] Connection header received
0811 03:15:00.847308 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:00.847318 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:00.847755 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:00.848045 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:00.849446 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:00.850719 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:00.852008 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:00.852757 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:00.852927 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:00.860362 (+ 7435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:00.860371 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:00.860389 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:00.860430 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:00.863713 (+ 3283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:00.864446 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:00.864453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:00.864460 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:00.864624 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:00.865041 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:00.865047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:00.865049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:00.865755 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:00.865980 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:00.866502 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 03:15:00.866912 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":376,"threads_started":1}
I20260811 03:15:00.867746 28617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:00.866951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:44637 (local address 127.11.60.3:57387)
0811 03:15:00.867470 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:00.867587 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":291,"threads_started":1}
I20260811 03:15:00.868717 28249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:00.955526 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 28073
I20260811 03:15:00.976212 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 28257
I20260811 03:15:00.994560 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:00.977276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.11.60.2:45407)
0811 03:15:00.977872 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:00.977887 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:00.977925 (+ 38us) server_negotiation.cc:407] Connection header received
0811 03:15:00.979759 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:00.979768 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:00.980205 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:00.980519 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:00.981917 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:00.983279 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:00.984697 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:00.985382 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:00.985531 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:00.988262 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:00.988270 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:00.988286 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:00.988337 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:00.991416 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:00.992122 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:00.992130 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:00.992137 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:00.992304 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:00.992646 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:00.992649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:00.992651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:00.993143 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:00.993363 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:00.993756 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 03:15:00.994183 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":280,"threads_started":1}
I20260811 03:15:00.995945 28403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:01.082763 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 28257
I20260811 03:15:01.103770 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 28413
I20260811 03:15:01.121914 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:01.104878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.11.60.3:37485)
0811 03:15:01.105530 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:01.105545 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:01.105588 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:15:01.107245 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:01.107255 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:01.107718 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:01.108013 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:01.109489 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:01.110796 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:01.112131 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:01.112776 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:01.112903 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:01.115752 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:01.115760 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:01.115775 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:01.115813 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:01.118818 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:01.119503 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:01.119510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:01.119516 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:01.119692 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:01.120031 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:01.120035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:01.120037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:01.120570 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:01.120801 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:01.121234 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 03:15:01.121612 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":307,"threads_started":1}
I20260811 03:15:01.123780 28556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:01.214439 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 28413
2026-08-11T03:15:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25306 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:15:01 dist-test-slave-tm1g krb5kdc[28625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:15:01 dist-test-slave-tm1g krb5kdc[28625](info): set up 2 sockets
Aug 11 03:15:01 dist-test-slave-tm1g krb5kdc[28625](info): commencing operation
krb5kdc: starting...
W20260811 03:15:02.389041 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:15:02 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:15:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:15:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:15:02.682734 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40559
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:40559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:15:03.025954 28639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:03.026238 28639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:03.026398 28639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:03.033519 28639 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 03:15:03.033622 28639 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 03:15:03.033661 28639 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 03:15:03.033715 28639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:15:03.033771 28639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:03.033857 28639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:03.033908 28639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:15:03.033958 28639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:15:03.043807 28639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:40559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:03.045626 28639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:03.047556 28639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:03.063592 28644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:03.065658 28646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:03.066608 28648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:03.066923 28639 server_base.cc:1034] running on GCE node
I20260811 03:15:03.068188 28639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:03.070387 28639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:03.071641 28639 hybrid_clock.cc:648] HybridClock initialized: now 1786418103071559 us; error 49 us; skew 500 ppm
I20260811 03:15:03.072144 28639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:03 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418103, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:03.076712 28639 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:15:03.079732 28639 webserver.cc:460] Webserver started at http://127.11.60.62:34779/ using document root <none> and password file <none>
I20260811 03:15:03.080816 28639 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:03.080958 28639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:03.081408 28639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:15:03.086411 28639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "bc513fcd6bde4c82a617037dc589ad95"
format_stamp: "Formatted at 2026-08-11 03:15:03 on dist-test-slave-tm1g"
server_key: "0be2d762f7247deb56c87cde34daa665"
server_key_iv: "29e3519c34728150c1481eee6f139628"
server_key_version: "encryptionkey@0"
I20260811 03:15:03.087399 28639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "bc513fcd6bde4c82a617037dc589ad95"
format_stamp: "Formatted at 2026-08-11 03:15:03 on dist-test-slave-tm1g"
server_key: "0be2d762f7247deb56c87cde34daa665"
server_key_iv: "29e3519c34728150c1481eee6f139628"
server_key_version: "encryptionkey@0"
I20260811 03:15:03.095223 28639 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260811 03:15:03.099040 28657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:03.100582 28639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:15:03.100770 28639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "bc513fcd6bde4c82a617037dc589ad95"
format_stamp: "Formatted at 2026-08-11 03:15:03 on dist-test-slave-tm1g"
server_key: "0be2d762f7247deb56c87cde34daa665"
server_key_iv: "29e3519c34728150c1481eee6f139628"
server_key_version: "encryptionkey@0"
I20260811 03:15:03.100992 28639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:03.132498 28639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:03.137073 28639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:03.137362 28639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:03.156848 28639 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:40559
I20260811 03:15:03.156867 28713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:40559 every 8 connection(s)
I20260811 03:15:03.158322 28639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:15:03.161494 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 28639
I20260811 03:15:03.161747 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:15:03.162397 28714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:15:03.162576 11504 external_mini_cluster.cc:1428] Setting key 21c8fd48dd0e57c17ce256f41ef08c4f
I20260811 03:15:03.177301 28714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Bootstrap starting.
Aug 11 03:15:03 dist-test-slave-tm1g krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:15:03.181231 28714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Neither blocks nor log segments found. Creating new log.
I20260811 03:15:03.182674 28714 log.cc:824] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Log is configured to *not* fsync() on all Append() calls
I20260811 03:15:03.186182 28714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: No bootstrap required, opened a new log
I20260811 03:15:03.188283 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:03.164583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56436 (local address 127.11.60.62:40559)
0811 03:15:03.165426 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:03.165437 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:03.165479 (+ 42us) server_negotiation.cc:407] Connection header received
0811 03:15:03.167171 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:03.167193 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:03.167594 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:03.167933 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:03.169939 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:03.171316 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:03.173316 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:03.174054 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:03.174202 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:03.179265 (+ 5063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:03.179277 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:03.179295 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:03.179341 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:03.183587 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:03.184460 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:03.184468 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:03.184477 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:03.184707 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:03.185070 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:03.185075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:03.185078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:03.185828 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:03.186125 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:03.186468 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 03:15:03.186942 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":413,"threads_started":1}
I20260811 03:15:03.194943 28714 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:03.195291 28714 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:15:03.195364 28714 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc513fcd6bde4c82a617037dc589ad95, State: Initialized, Role: FOLLOWER
I20260811 03:15:03.195892 28714 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:03.196103 28714 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:15:03.196203 28714 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:15:03.196326 28714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:15:03.198787 28714 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:03.199231 28714 leader_election.cc:302] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc513fcd6bde4c82a617037dc589ad95; no voters:
I20260811 03:15:03.199786 28714 leader_election.cc:288] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:15:03.199971 28719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:15:03.200647 28719 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 1 LEADER]: Becoming Leader. State: Replica: bc513fcd6bde4c82a617037dc589ad95, State: Running, Role: LEADER
I20260811 03:15:03.201303 28719 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:03.202055 28714 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:15:03.204505 28720 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc513fcd6bde4c82a617037dc589ad95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } } }
I20260811 03:15:03.205195 28720 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: This master's current role is: LEADER
I20260811 03:15:03.204535 28721 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc513fcd6bde4c82a617037dc589ad95. Latest consensus state: current_term: 1 leader_uuid: "bc513fcd6bde4c82a617037dc589ad95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } } }
I20260811 03:15:03.205859 28721 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: This master's current role is: LEADER
W20260811 03:15:03.214197 28736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:15:03.214358 28736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:15:03.214496 28729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:15:03.216742 28729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:15:03.225770 28729 catalog_manager.cc:1347] Generated new cluster ID: 427e50831f6549ef86f1aed7575a85af
I20260811 03:15:03.225886 28729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:15:03.261193 28729 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:15:03.263187 28729 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:15:03.273686 28729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Generated new TSK 0
I20260811 03:15:03.274819 28729 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:15:03.366466 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:40559
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:15:03.715197 28742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:03.715473 28742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:03.715569 28742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:03.722801 28742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:03.722966 28742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:03.723097 28742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:15:03.733184 28742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:03.734616 28742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:03.735911 28742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:03.750981 28748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:03.750937 28749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:03.751010 28752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:03.752389 28742 server_base.cc:1034] running on GCE node
I20260811 03:15:03.753089 28742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:03.754053 28742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:03.755275 28742 hybrid_clock.cc:648] HybridClock initialized: now 1786418103755216 us; error 70 us; skew 500 ppm
I20260811 03:15:03.755595 28742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:03 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418103, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:03.758711 28742 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:15:03.760123 28742 webserver.cc:460] Webserver started at http://127.11.60.1:44257/ using document root <none> and password file <none>
I20260811 03:15:03.760839 28742 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:03.760979 28742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:03.761297 28742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:15:03.764315 28742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "a5aa310eaad0427484df888834585d8b"
format_stamp: "Formatted at 2026-08-11 03:15:03 on dist-test-slave-tm1g"
server_key: "061ef81b9098058979b2fd5015e2e19a"
server_key_iv: "42a637313d6fd381aad82b86aa3ff868"
server_key_version: "encryptionkey@0"
I20260811 03:15:03.765094 28742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "a5aa310eaad0427484df888834585d8b"
format_stamp: "Formatted at 2026-08-11 03:15:03 on dist-test-slave-tm1g"
server_key: "061ef81b9098058979b2fd5015e2e19a"
server_key_iv: "42a637313d6fd381aad82b86aa3ff868"
server_key_version: "encryptionkey@0"
I20260811 03:15:03.770292 28742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:15:03.773891 28761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:03.775350 28742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:15:03.775535 28742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "a5aa310eaad0427484df888834585d8b"
format_stamp: "Formatted at 2026-08-11 03:15:03 on dist-test-slave-tm1g"
server_key: "061ef81b9098058979b2fd5015e2e19a"
server_key_iv: "42a637313d6fd381aad82b86aa3ff868"
server_key_version: "encryptionkey@0"
I20260811 03:15:03.775755 28742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:03.792711 28742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:03.796833 28742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:03.797132 28742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:03.798107 28742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:15:03.799479 28742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:15:03.799580 28742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:03.799723 28742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:15:03.799791 28742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:03.833379 28742 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:44885
I20260811 03:15:03.833415 28891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:44885 every 8 connection(s)
I20260811 03:15:03.834694 28742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:15:03.835125 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 28742
I20260811 03:15:03.835357 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:15:03.836251 11504 external_mini_cluster.cc:1428] Setting key 2c34d231bab22fa35398d77a3fc8cbb0
Aug 11 03:15:03 dist-test-slave-tm1g krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418103, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:15:03.854984 28894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:03.837459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:34103 (local address 127.11.60.62:40559)
0811 03:15:03.837943 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:03.837948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:03.839108 (+ 1160us) server_negotiation.cc:407] Connection header received
0811 03:15:03.840912 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:03.840917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:03.841003 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:03.841131 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:03.842691 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:03.843432 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:03.845003 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:03.845428 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:03.845563 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:03.849036 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:03.849043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:03.849052 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:03.849085 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:03.851782 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:03.852484 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:03.852490 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:03.852494 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:03.852625 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:03.853121 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:03.853126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:03.853130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:03.853403 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:03.853524 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:03.854512 (+ 988us) server_negotiation.cc:299] Negotiation successful
0811 03:15:03.854713 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":251,"threads_started":1}
I20260811 03:15:03.856609 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:03.838150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:34103)
0811 03:15:03.838969 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:03.839006 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:03.840655 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:03.841367 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:03.841376 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:03.841894 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:15:03.842525 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:03.842544 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:03.843602 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:03.843606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:03.844854 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:03.844866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:03.845689 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:03.845698 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:03.845911 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:03.846652 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:15:03.846681 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:15:03.848816 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:15:03.851913 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:03.851921 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:03.851938 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:03.852353 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:03.852735 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:03.852740 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:03.852744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:03.852947 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:03.853548 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:15:03.853559 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:15:03.854051 (+ 492us) client_negotiation.cc:769] Sending connection context
0811 03:15:03.854420 (+ 369us) client_negotiation.cc:240] Negotiation successful
0811 03:15:03.855046 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":381,"threads_started":1}
I20260811 03:15:03.858363 28892 heartbeater.cc:347] Connected to a master server at 127.11.60.62:40559
I20260811 03:15:03.858697 28892 heartbeater.cc:464] Registering TS with master...
I20260811 03:15:03.859427 28892 heartbeater.cc:511] Master 127.11.60.62:40559 requested a full tablet report, sending...
I20260811 03:15:03.861553 28677 ts_manager.cc:194] Registered new tserver with Master: a5aa310eaad0427484df888834585d8b (127.11.60.1:44885)
I20260811 03:15:03.863101 28677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:34103
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:15:03.906381 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:40559
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:15:04.250954 28899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:04.251246 28899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:04.251415 28899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:04.258876 28899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:04.258980 28899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:04.259157 28899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:15:04.269104 28899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:04.270500 28899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:04.271831 28899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:04.285912 28905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:04.285867 28906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:04.285949 28909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:04.286834 28899 server_base.cc:1034] running on GCE node
I20260811 03:15:04.287452 28899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:04.288408 28899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:04.289633 28899 hybrid_clock.cc:648] HybridClock initialized: now 1786418104289591 us; error 54 us; skew 500 ppm
I20260811 03:15:04.289942 28899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:04 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418104, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:04.293174 28899 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:15:04.294798 28899 webserver.cc:460] Webserver started at http://127.11.60.2:41803/ using document root <none> and password file <none>
I20260811 03:15:04.295533 28899 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:04.295640 28899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:04.295905 28899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:15:04.298879 28899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "0ec3665007a342409d413d1ae84d7e5d"
format_stamp: "Formatted at 2026-08-11 03:15:04 on dist-test-slave-tm1g"
server_key: "ccd7859c07cf0f964e997259c48c9977"
server_key_iv: "847eb9a85670cd0c58b52bb7df693c1f"
server_key_version: "encryptionkey@0"
I20260811 03:15:04.299600 28899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "0ec3665007a342409d413d1ae84d7e5d"
format_stamp: "Formatted at 2026-08-11 03:15:04 on dist-test-slave-tm1g"
server_key: "ccd7859c07cf0f964e997259c48c9977"
server_key_iv: "847eb9a85670cd0c58b52bb7df693c1f"
server_key_version: "encryptionkey@0"
I20260811 03:15:04.304993 28899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:15:04.308779 28917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:04.310050 28899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:15:04.310186 28899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "0ec3665007a342409d413d1ae84d7e5d"
format_stamp: "Formatted at 2026-08-11 03:15:04 on dist-test-slave-tm1g"
server_key: "ccd7859c07cf0f964e997259c48c9977"
server_key_iv: "847eb9a85670cd0c58b52bb7df693c1f"
server_key_version: "encryptionkey@0"
I20260811 03:15:04.310402 28899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:04.322656 28899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:04.327118 28899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:04.327376 28899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:04.328286 28899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:15:04.329574 28899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:15:04.329653 28899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:04.329736 28899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:15:04.329787 28899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:04.364874 28899 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:38241
I20260811 03:15:04.364930 29042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:38241 every 8 connection(s)
I20260811 03:15:04.366273 28899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:15:04.376474 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 28899
I20260811 03:15:04.376693 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:15:04.377182 11504 external_mini_cluster.cc:1428] Setting key e6fdafb62de525bc64b35873eea6b35d
Aug 11 03:15:04 dist-test-slave-tm1g krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418104, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:15:04.387050 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.368890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:49773 (local address 127.11.60.62:40559)
0811 03:15:04.369363 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:04.369368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:04.370185 (+ 817us) server_negotiation.cc:407] Connection header received
0811 03:15:04.372088 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:04.372095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:04.372184 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:04.372300 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:04.373810 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.374626 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.376186 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.376698 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.376839 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.381078 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:04.381083 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:04.381086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:04.381120 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:04.383849 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:04.384627 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:04.384632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:04.384636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:04.384773 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:04.385201 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:04.385207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:04.385211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:04.385516 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:04.385671 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:04.386412 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 03:15:04.386667 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":252,"threads_started":1}
I20260811 03:15:04.388031 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.369302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:49773)
0811 03:15:04.370012 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:04.370053 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:04.371798 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:04.372551 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:04.372560 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:04.373075 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:15:04.373637 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:04.373655 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.374785 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.374789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:04.376031 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:04.376045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.376949 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.376956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:04.377183 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.378198 (+ 1015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:15:04.378230 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:15:04.380839 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:15:04.384017 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:04.384026 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:04.384046 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:04.384503 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:04.384883 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:04.384889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:04.384892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:04.385090 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:04.385644 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:15:04.385652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:15:04.386022 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 03:15:04.386299 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 03:15:04.386909 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":271,"threads_started":1}
I20260811 03:15:04.389617 29043 heartbeater.cc:347] Connected to a master server at 127.11.60.62:40559
I20260811 03:15:04.389922 29043 heartbeater.cc:464] Registering TS with master...
I20260811 03:15:04.390725 29043 heartbeater.cc:511] Master 127.11.60.62:40559 requested a full tablet report, sending...
I20260811 03:15:04.392220 28677 ts_manager.cc:194] Registered new tserver with Master: 0ec3665007a342409d413d1ae84d7e5d (127.11.60.2:38241)
I20260811 03:15:04.393080 28677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:49773
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:15:04.441254 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:40559
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:15:04.789407 29050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:04.789760 29050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:04.789928 29050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:04.797318 29050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:04.797490 29050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:04.797668 29050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:15:04.808130 29050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:04.809554 29050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:04.811019 29050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:04.824406 29056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:04.824440 29055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:04.824409 29059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:04.825063 29050 server_base.cc:1034] running on GCE node
I20260811 03:15:04.825798 29050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:04.826810 29050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:04.828039 29050 hybrid_clock.cc:648] HybridClock initialized: now 1786418104827989 us; error 60 us; skew 500 ppm
I20260811 03:15:04.828363 29050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:04 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418104, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:04.831713 29050 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:15:04.833252 29050 webserver.cc:460] Webserver started at http://127.11.60.3:45331/ using document root <none> and password file <none>
I20260811 03:15:04.834007 29050 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:04.834139 29050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:04.834517 29050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:15:04.837584 29050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "b6d4833e8a5f4f55b31791ffe41dab2b"
format_stamp: "Formatted at 2026-08-11 03:15:04 on dist-test-slave-tm1g"
server_key: "974b19d296a9889b121736b39e11faae"
server_key_iv: "8343c29726352ca75253fab07c83285b"
server_key_version: "encryptionkey@0"
I20260811 03:15:04.838449 29050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "b6d4833e8a5f4f55b31791ffe41dab2b"
format_stamp: "Formatted at 2026-08-11 03:15:04 on dist-test-slave-tm1g"
server_key: "974b19d296a9889b121736b39e11faae"
server_key_iv: "8343c29726352ca75253fab07c83285b"
server_key_version: "encryptionkey@0"
I20260811 03:15:04.843895 29050 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 03:15:04.847627 29066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:04.849076 29050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 03:15:04.849272 29050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "b6d4833e8a5f4f55b31791ffe41dab2b"
format_stamp: "Formatted at 2026-08-11 03:15:04 on dist-test-slave-tm1g"
server_key: "974b19d296a9889b121736b39e11faae"
server_key_iv: "8343c29726352ca75253fab07c83285b"
server_key_version: "encryptionkey@0"
I20260811 03:15:04.849506 29050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:04.863217 29050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:04.866140 28892 heartbeater.cc:503] Master 127.11.60.62:40559 was elected leader, sending a full tablet report...
I20260811 03:15:04.867856 29050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:04.868227 29050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:04.869239 29050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:15:04.870682 29050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:15:04.870757 29050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:04.870848 29050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:15:04.870898 29050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:04.905707 29050 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:43271
I20260811 03:15:04.905743 29190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:43271 every 8 connection(s)
I20260811 03:15:04.907171 29050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:15:04.909677 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 29050
I20260811 03:15:04.909873 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:15:04.910490 11504 external_mini_cluster.cc:1428] Setting key bd6133f8bc83a2b1383d1c99b43bd084
Aug 11 03:15:04 dist-test-slave-tm1g krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418104, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:15:04.927676 29193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.909786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:48937 (local address 127.11.60.62:40559)
0811 03:15:04.910339 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:04.910346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:04.911457 (+ 1111us) server_negotiation.cc:407] Connection header received
0811 03:15:04.913796 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:04.913803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:04.913879 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:04.913970 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:04.915559 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.916324 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.917827 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.918211 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.918357 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.921643 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:04.921648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:04.921651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:04.921683 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:04.924370 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:04.925219 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:04.925225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:04.925232 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:04.925416 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:04.925854 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:04.925859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:04.925862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:04.926146 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:04.926334 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:04.927017 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 03:15:04.927279 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":266,"threads_started":1}
I20260811 03:15:04.928691 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.910448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:48937)
0811 03:15:04.911268 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:04.911334 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:04.913511 (+ 2177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:04.914219 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:04.914228 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:04.914837 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:15:04.915411 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:04.915428 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.916454 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.916457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:04.917704 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:04.917716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.918447 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.918452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:04.918652 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.919273 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:15:04.919293 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:15:04.921434 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:15:04.924508 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:04.924517 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:04.924534 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:04.925093 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:04.925521 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:04.925526 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:04.925529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:04.925732 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:04.926290 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:15:04.926314 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:15:04.926651 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 03:15:04.926916 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 03:15:04.927451 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":318,"threads_started":1}
I20260811 03:15:04.929986 29191 heartbeater.cc:347] Connected to a master server at 127.11.60.62:40559
I20260811 03:15:04.930341 29191 heartbeater.cc:464] Registering TS with master...
I20260811 03:15:04.931139 29191 heartbeater.cc:511] Master 127.11.60.62:40559 requested a full tablet report, sending...
I20260811 03:15:04.932583 28677 ts_manager.cc:194] Registered new tserver with Master: b6d4833e8a5f4f55b31791ffe41dab2b (127.11.60.3:43271)
I20260811 03:15:04.933404 28677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:48937
I20260811 03:15:04.938102 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:15:04.955928 29193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.942683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56444 (local address 127.11.60.62:40559)
0811 03:15:04.942836 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:04.942841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:04.943351 (+ 510us) server_negotiation.cc:407] Connection header received
0811 03:15:04.943472 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:04.943476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:04.943543 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:04.943637 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:04.945104 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.945912 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.947473 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.947873 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.947974 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.950489 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:04.950494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:04.950497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:04.950531 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:04.953207 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:04.954075 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:04.954080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:04.954082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:04.954213 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:04.954629 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:04.954633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:04.954635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:04.954957 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:04.955237 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:04.955518 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 03:15:04.955695 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 03:15:04.960062 28677 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:15:04.967108 28677 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:15:04.991603 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.980771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.11.60.3:43271)
0811 03:15:04.981866 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:04.981873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:04.982378 (+ 505us) server_negotiation.cc:407] Connection header received
0811 03:15:04.982693 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:04.982700 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:04.982994 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:04.983234 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:15:04.984507 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.985701 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.988481 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.990017 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.990115 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.990397 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:04.990973 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 03:15:04.991159 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":947,"thread_start_us":778,"threads_started":1}
I20260811 03:15:04.991991 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.980120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:43271 (local address 127.0.0.1:51764)
0811 03:15:04.982236 (+ 2116us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:04.982251 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:04.982439 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:04.983560 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:04.983566 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:04.983616 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:15:04.984012 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:04.984027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.985874 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.985878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:04.988264 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:04.988275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.990242 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.990249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:04.990575 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.990737 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 03:15:04.990905 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 03:15:04.991256 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2017,"mutex_wait_us":194,"thread_start_us":536,"threads_started":1}
I20260811 03:15:04.995661 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.983031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54434 (local address 127.11.60.2:38241)
0811 03:15:04.986834 (+ 3803us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:04.986843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:04.986880 (+ 37us) server_negotiation.cc:407] Connection header received
0811 03:15:04.986991 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:04.987000 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:04.987487 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:04.988518 (+ 1031us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:15:04.988753 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.990259 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.993052 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.994554 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.994692 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.994883 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:04.995136 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 03:15:04.995308 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3630,"thread_start_us":3478,"threads_started":1}
I20260811 03:15:04.995651 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.982575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:38241 (local address 127.0.0.1:54434)
0811 03:15:04.983133 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:04.983153 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:04.983265 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:04.987748 (+ 4483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:04.987753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:04.987780 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:15:04.988262 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:04.988275 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.990444 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.990457 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:04.992822 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:04.992835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.994756 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.994763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:04.994925 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.994966 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 03:15:04.995087 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 03:15:04.995225 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":77,"thread_start_us":657,"threads_started":1}
I20260811 03:15:04.995988 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.979952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:44885 (local address 127.0.0.1:39490)
0811 03:15:04.980849 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:04.980899 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:04.981068 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:04.987928 (+ 6860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:04.987937 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:04.987965 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:15:04.988488 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:04.988501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.990060 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.990068 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:04.992768 (+ 2700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:04.992786 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.995085 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.995091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:04.995271 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.995302 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 03:15:04.995399 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 03:15:04.995546 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":1,"thread_start_us":2041,"threads_started":1}
I20260811 03:15:04.996625 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:04.980109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39490 (local address 127.11.60.1:44885)
0811 03:15:04.987315 (+ 7206us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:04.987322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:04.987343 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:15:04.987416 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:04.987443 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:04.987687 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:04.988691 (+ 1004us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:15:04.988894 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.989910 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.992952 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:04.994557 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:04.994692 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:04.994922 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:04.996226 (+ 1304us) server_negotiation.cc:299] Negotiation successful
0811 03:15:04.996392 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7024,"thread_start_us":6884,"threads_started":1}
I20260811 03:15:04.996510 29117 tablet_service.cc:1467] Processing CreateTablet for tablet 7a55db5550f845c1bda8077ce3101cdb (DEFAULT_TABLE table=test-table [id=7d23fd1dcca8416daef4bb40c2ba7c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:15:04.998127 29117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a55db5550f845c1bda8077ce3101cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:15:05.000396 28970 tablet_service.cc:1467] Processing CreateTablet for tablet 7a55db5550f845c1bda8077ce3101cdb (DEFAULT_TABLE table=test-table [id=7d23fd1dcca8416daef4bb40c2ba7c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:15:05.000399 28815 tablet_service.cc:1467] Processing CreateTablet for tablet 7a55db5550f845c1bda8077ce3101cdb (DEFAULT_TABLE table=test-table [id=7d23fd1dcca8416daef4bb40c2ba7c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:15:05.001895 28970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a55db5550f845c1bda8077ce3101cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:15:05.001886 28815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a55db5550f845c1bda8077ce3101cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:15:05.010174 29212 tablet_bootstrap.cc:492] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: Bootstrap starting.
I20260811 03:15:05.014220 29212 tablet_bootstrap.cc:654] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: Neither blocks nor log segments found. Creating new log.
I20260811 03:15:05.015465 29212 log.cc:824] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: Log is configured to *not* fsync() on all Append() calls
I20260811 03:15:05.019057 29214 tablet_bootstrap.cc:492] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: Bootstrap starting.
I20260811 03:15:05.019776 29212 tablet_bootstrap.cc:492] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: No bootstrap required, opened a new log
I20260811 03:15:05.020037 29215 tablet_bootstrap.cc:492] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: Bootstrap starting.
I20260811 03:15:05.020064 29212 ts_tablet_manager.cc:1397] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260811 03:15:05.022903 29214 tablet_bootstrap.cc:654] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: Neither blocks nor log segments found. Creating new log.
I20260811 03:15:05.023592 29215 tablet_bootstrap.cc:654] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: Neither blocks nor log segments found. Creating new log.
I20260811 03:15:05.024242 29214 log.cc:824] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: Log is configured to *not* fsync() on all Append() calls
I20260811 03:15:05.024845 29215 log.cc:824] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: Log is configured to *not* fsync() on all Append() calls
I20260811 03:15:05.027514 29214 tablet_bootstrap.cc:492] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: No bootstrap required, opened a new log
I20260811 03:15:05.027809 29214 ts_tablet_manager.cc:1397] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 03:15:05.028271 29212 raft_consensus.cc:348] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:05.028632 29212 raft_consensus.cc:374] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:15:05.028756 29212 raft_consensus.cc:729] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d4833e8a5f4f55b31791ffe41dab2b, State: Initialized, Role: FOLLOWER
I20260811 03:15:05.028967 29215 tablet_bootstrap.cc:492] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: No bootstrap required, opened a new log
I20260811 03:15:05.029245 29215 ts_tablet_manager.cc:1397] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 03:15:05.029394 29212 consensus_queue.cc:260] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:05.030511 29191 heartbeater.cc:503] Master 127.11.60.62:40559 was elected leader, sending a full tablet report...
I20260811 03:15:05.031157 29212 ts_tablet_manager.cc:1428] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260811 03:15:05.038847 29214 raft_consensus.cc:348] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:05.039341 29214 raft_consensus.cc:374] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:15:05.039453 29214 raft_consensus.cc:729] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5aa310eaad0427484df888834585d8b, State: Initialized, Role: FOLLOWER
I20260811 03:15:05.039896 29215 raft_consensus.cc:348] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:05.040155 29214 consensus_queue.cc:260] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:05.040361 29215 raft_consensus.cc:374] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:15:05.040452 29215 raft_consensus.cc:729] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec3665007a342409d413d1ae84d7e5d, State: Initialized, Role: FOLLOWER
I20260811 03:15:05.041119 29215 consensus_queue.cc:260] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:05.042258 29043 heartbeater.cc:503] Master 127.11.60.62:40559 was elected leader, sending a full tablet report...
I20260811 03:15:05.042318 29214 ts_tablet_manager.cc:1428] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260811 03:15:05.042932 29215 ts_tablet_manager.cc:1428] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260811 03:15:05.059435 29220 raft_consensus.cc:482] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:15:05.059722 29220 raft_consensus.cc:504] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:05.061422 29220 leader_election.cc:288] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6d4833e8a5f4f55b31791ffe41dab2b (127.11.60.3:43271), 0ec3665007a342409d413d1ae84d7e5d (127.11.60.2:38241)
I20260811 03:15:05.069478 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:05.061925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:50939 (local address 127.11.60.2:38241)
0811 03:15:05.062158 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:05.062165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:05.062964 (+ 799us) server_negotiation.cc:407] Connection header received
0811 03:15:05.063205 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:05.063211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:05.063285 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:05.063374 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:15:05.064297 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.065474 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.067727 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.068503 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.068654 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:05.068777 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:05.069073 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 03:15:05.069201 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 03:15:05.069497 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:05.062000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:39143 (local address 127.11.60.3:43271)
0811 03:15:05.062178 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:05.062186 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:05.062963 (+ 777us) server_negotiation.cc:407] Connection header received
0811 03:15:05.063171 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:05.063177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:05.063257 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:05.063365 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:15:05.064297 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.065474 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.067736 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.068540 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.068697 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:05.068814 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:05.069102 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 03:15:05.069243 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 03:15:05.069661 29222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:05.061809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:43271 (local address 127.11.60.1:39143)
0811 03:15:05.062854 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:05.062873 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:05.063013 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:05.063402 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:05.063407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:05.063461 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:15:05.064105 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:05.064125 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.065624 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.065630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:05.067570 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:05.067584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.068779 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.068785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:05.068945 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:05.068967 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 03:15:05.069073 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 03:15:05.069196 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":145,"spinlock_wait_cycles":16256,"thread_start_us":578,"threads_started":1}
I20260811 03:15:05.070086 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:05.061711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:38241 (local address 127.11.60.1:50939)
0811 03:15:05.062802 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:05.062836 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:05.063072 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:05.063402 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:05.063407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:05.063460 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:15:05.064164 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:05.064173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.065664 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.065668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:05.067565 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:05.067579 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.068741 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.068747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:05.068902 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:05.068926 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:15:05.069026 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 03:15:05.069156 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":123,"spinlock_wait_cycles":83968,"thread_start_us":698,"threads_started":1}
I20260811 03:15:05.070859 29142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a55db5550f845c1bda8077ce3101cdb" candidate_uuid: "a5aa310eaad0427484df888834585d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" is_pre_election: true
I20260811 03:15:05.071060 28994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a55db5550f845c1bda8077ce3101cdb" candidate_uuid: "a5aa310eaad0427484df888834585d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec3665007a342409d413d1ae84d7e5d" is_pre_election: true
I20260811 03:15:05.071547 29142 raft_consensus.cc:2454] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5aa310eaad0427484df888834585d8b in term 0.
I20260811 03:15:05.071563 28994 raft_consensus.cc:2454] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5aa310eaad0427484df888834585d8b in term 0.
I20260811 03:15:05.072329 28763 leader_election.cc:302] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0ec3665007a342409d413d1ae84d7e5d, a5aa310eaad0427484df888834585d8b, b6d4833e8a5f4f55b31791ffe41dab2b; no voters:
I20260811 03:15:05.072764 29220 raft_consensus.cc:2781] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:15:05.072927 29220 raft_consensus.cc:482] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:15:05.073060 29220 raft_consensus.cc:3037] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:15:05.075349 29220 raft_consensus.cc:504] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:05.075970 29220 leader_election.cc:288] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [CANDIDATE]: Term 1 election: Requested vote from peers b6d4833e8a5f4f55b31791ffe41dab2b (127.11.60.3:43271), 0ec3665007a342409d413d1ae84d7e5d (127.11.60.2:38241)
I20260811 03:15:05.076431 29142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a55db5550f845c1bda8077ce3101cdb" candidate_uuid: "a5aa310eaad0427484df888834585d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b"
I20260811 03:15:05.076534 28994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a55db5550f845c1bda8077ce3101cdb" candidate_uuid: "a5aa310eaad0427484df888834585d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec3665007a342409d413d1ae84d7e5d"
I20260811 03:15:05.076642 29142 raft_consensus.cc:3037] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:15:05.076779 28994 raft_consensus.cc:3037] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:15:05.079046 29142 raft_consensus.cc:2454] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5aa310eaad0427484df888834585d8b in term 1.
I20260811 03:15:05.079137 28994 raft_consensus.cc:2454] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5aa310eaad0427484df888834585d8b in term 1.
I20260811 03:15:05.079479 28764 leader_election.cc:302] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5aa310eaad0427484df888834585d8b, b6d4833e8a5f4f55b31791ffe41dab2b; no voters:
I20260811 03:15:05.079823 29220 raft_consensus.cc:2781] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:15:05.080263 29220 raft_consensus.cc:686] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 1 LEADER]: Becoming Leader. State: Replica: a5aa310eaad0427484df888834585d8b, State: Running, Role: LEADER
I20260811 03:15:05.080785 29220 consensus_queue.cc:237] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:05.085944 28675 catalog_manager.cc:5495] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b reported cstate change: term changed from 0 to 1, leader changed from <none> to a5aa310eaad0427484df888834585d8b (127.11.60.1). New cstate: current_term: 1 leader_uuid: "a5aa310eaad0427484df888834585d8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } health_report { overall_health: UNKNOWN } } }
W20260811 03:15:05.087226 28893 tablet.cc:2369] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:15:05.118433 29045 tablet.cc:2369] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:15:05.122272 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:05.116994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39504 (local address 127.11.60.1:44885)
0811 03:15:05.117194 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:05.117199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:05.117219 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:15:05.117311 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:05.117315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:05.117393 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:05.117522 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:15:05.117845 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.118569 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.120229 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.120647 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.120739 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:05.121158 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:15:05.121688 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:15:05.121827 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:05.121915 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 03:15:05.121985 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 03:15:05.130820 29142 raft_consensus.cc:1261] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b [term 1 FOLLOWER]: Refusing update from remote peer a5aa310eaad0427484df888834585d8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:15:05.130821 28994 raft_consensus.cc:1261] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d [term 1 FOLLOWER]: Refusing update from remote peer a5aa310eaad0427484df888834585d8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:15:05.131914 29224 consensus_queue.cc:1035] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:15:05.132292 29220 consensus_queue.cc:1035] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:15:05.145658 29228 mvcc.cc:205] Tried to move back new op lower bound from 7317168558604410880 to 7317168558414909440. Current Snapshot: MvccSnapshot[applied={T|T < 7317168558604410880}]
I20260811 03:15:05.179533 28675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:15:05.179916 28675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:15:05.184706 28675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Sending DeleteTablet for 3 replicas of tablet 7a55db5550f845c1bda8077ce3101cdb
I20260811 03:15:05.185920 28815 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a55db5550f845c1bda8077ce3101cdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:15:05 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:39490
I20260811 03:15:05.186288 29117 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a55db5550f845c1bda8077ce3101cdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:15:05 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:51764
I20260811 03:15:05.187425 28970 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a55db5550f845c1bda8077ce3101cdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:15:05 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:54434
I20260811 03:15:05.188771 29239 tablet_replica.cc:330] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: stopping tablet replica
I20260811 03:15:05.188776 29116 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a55db5550f845c1bda8077ce3101cdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:15:05 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:51764
I20260811 03:15:05.189368 29239 raft_consensus.cc:2229] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:15:05.189769 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 28742
I20260811 03:15:05.190034 29239 raft_consensus.cc:2258] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:15:05.190899 29238 tablet_replica.cc:330] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: stopping tablet replica
I20260811 03:15:05.191403 29238 raft_consensus.cc:2229] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:15:05.191989 29238 raft_consensus.cc:2258] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:15:05.194679 29238 ts_tablet_manager.cc:1905] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:15:05.194937 29237 tablet_replica.cc:330] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: stopping tablet replica
I20260811 03:15:05.195478 29237 raft_consensus.cc:2229] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:15:05.196311 29237 raft_consensus.cc:2258] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:15:05.198496 29239 ts_tablet_manager.cc:1905] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:15:05.199942 29238 ts_tablet_manager.cc:1918] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:15:05.200134 29238 log.cc:1196] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/7a55db5550f845c1bda8077ce3101cdb
I20260811 03:15:05.200709 29238 ts_tablet_manager.cc:1939] T 7a55db5550f845c1bda8077ce3101cdb P b6d4833e8a5f4f55b31791ffe41dab2b: Deleting consensus metadata
I20260811 03:15:05.204202 29237 ts_tablet_manager.cc:1905] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:15:05.206182 29239 ts_tablet_manager.cc:1918] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:15:05.206395 29239 log.cc:1196] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/7a55db5550f845c1bda8077ce3101cdb
I20260811 03:15:05.206830 28660 catalog_manager.cc:4841] TS b6d4833e8a5f4f55b31791ffe41dab2b (127.11.60.3:43271): tablet 7a55db5550f845c1bda8077ce3101cdb (table test-table [id=7d23fd1dcca8416daef4bb40c2ba7c12]) successfully deleted
I20260811 03:15:05.206992 29239 ts_tablet_manager.cc:1939] T 7a55db5550f845c1bda8077ce3101cdb P 0ec3665007a342409d413d1ae84d7e5d: Deleting consensus metadata
W20260811 03:15:05.207666 28660 catalog_manager.cc:4808] TS b6d4833e8a5f4f55b31791ffe41dab2b (127.11.60.3:43271): delete failed for tablet 7a55db5550f845c1bda8077ce3101cdb because the tablet was not found. No further retry: Not found: Tablet not found: 7a55db5550f845c1bda8077ce3101cdb
I20260811 03:15:05.208902 28659 catalog_manager.cc:4841] TS 0ec3665007a342409d413d1ae84d7e5d (127.11.60.2:38241): tablet 7a55db5550f845c1bda8077ce3101cdb (table test-table [id=7d23fd1dcca8416daef4bb40c2ba7c12]) successfully deleted
Aug 11 03:15:05 dist-test-slave-tm1g krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:15:05.213547 29237 ts_tablet_manager.cc:1918] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:15:05.213748 29237 log.cc:1196] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/7a55db5550f845c1bda8077ce3101cdb
I20260811 03:15:05.214363 29237 ts_tablet_manager.cc:1939] T 7a55db5550f845c1bda8077ce3101cdb P a5aa310eaad0427484df888834585d8b: Deleting consensus metadata
I20260811 03:15:05.216341 28659 catalog_manager.cc:4841] TS a5aa310eaad0427484df888834585d8b (127.11.60.1:44885): tablet 7a55db5550f845c1bda8077ce3101cdb (table test-table [id=7d23fd1dcca8416daef4bb40c2ba7c12]) successfully deleted
I20260811 03:15:05.221560 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:05.190972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39510 (local address 127.11.60.1:44885)
0811 03:15:05.191175 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:05.191181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:05.200840 (+ 9659us) server_negotiation.cc:407] Connection header received
0811 03:15:05.201185 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:05.201191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:05.201293 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:05.201417 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:05.203810 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.205122 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.209391 (+ 4269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.209899 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.210072 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:05.214816 (+ 4744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:05.214829 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:05.214844 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:05.214884 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:05.218254 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:05.219243 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:05.219253 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:05.219257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:05.219449 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:05.219861 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:05.219867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:05.219870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:05.220358 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:05.220723 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:05.220908 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 03:15:05.221173 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 03:15:05.222441 28887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:05.322732 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 28742
I20260811 03:15:05.344537 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 28899
Aug 11 03:15:05 dist-test-slave-tm1g krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:15:05.361438 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:05.345363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54442 (local address 127.11.60.2:38241)
0811 03:15:05.345598 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:05.345603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:05.345625 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:15:05.345811 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:05.345817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:05.345894 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:05.346005 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:05.347467 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.348207 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.349697 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.350109 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.350243 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:05.355013 (+ 4770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:05.355028 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:05.355044 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:05.355081 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:05.358160 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:05.359198 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:05.359207 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:05.359209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:05.359345 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:05.359770 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:05.359773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:05.359775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:05.360202 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:05.360526 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:05.360839 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 03:15:05.361081 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 03:15:05.362147 29037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:05.453171 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 28899
I20260811 03:15:05.472987 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 29050
Aug 11 03:15:05 dist-test-slave-tm1g krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:15:05.490394 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:05.473959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51770 (local address 127.11.60.3:43271)
0811 03:15:05.474214 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:05.474219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:05.474283 (+ 64us) server_negotiation.cc:407] Connection header received
0811 03:15:05.474421 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:05.474425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:05.474498 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:05.474600 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:05.475992 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.476811 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.478260 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:05.478804 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:05.478908 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:05.484141 (+ 5233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:05.484159 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:05.484177 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:05.484226 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:05.487402 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:05.488265 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:05.488273 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:05.488275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:05.488429 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:05.488859 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:05.488863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:05.488865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:05.489266 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:05.489480 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:05.489876 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 03:15:05.490057 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260811 03:15:05.491114 29185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:05.584115 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 29050
I20260811 03:15:05.604233 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 28639
I20260811 03:15:05.605557 28708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:05.719434 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 28639
I20260811 03:15:25.748136 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40559
--webserver_interface=127.11.60.62
--webserver_port=34779
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:40559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:15:26.098538 29245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:26.098835 29245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:26.098995 29245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:26.106355 29245 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 03:15:26.106467 29245 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 03:15:26.106544 29245 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 03:15:26.106622 29245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:15:26.106719 29245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:26.106833 29245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:26.106928 29245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:15:26.107009 29245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:15:26.117364 29245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:40559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=34779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:26.118781 29245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:26.120230 29245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:26.147431 29251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:26.147514 29252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:26.147503 29254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:26.147821 29245 server_base.cc:1034] running on GCE node
I20260811 03:15:26.149092 29245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:26.150767 29245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:26.152053 29245 hybrid_clock.cc:648] HybridClock initialized: now 1786418126151980 us; error 37 us; skew 500 ppm
I20260811 03:15:26.152498 29245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:26 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418126, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:26.156268 29245 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:15:26.157918 29245 webserver.cc:460] Webserver started at http://127.11.60.62:34779/ using document root <none> and password file <none>
I20260811 03:15:26.158799 29245 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:26.158906 29245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:26.165023 29245 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 03:15:26.168231 29262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:26.169785 29245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 03:15:26.169945 29245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "bc513fcd6bde4c82a617037dc589ad95"
format_stamp: "Formatted at 2026-08-11 03:15:03 on dist-test-slave-tm1g"
server_key: "0be2d762f7247deb56c87cde34daa665"
server_key_iv: "29e3519c34728150c1481eee6f139628"
server_key_version: "encryptionkey@0"
I20260811 03:15:26.171092 29245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:26.190613 29245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:26.195247 29245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:26.195529 29245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:26.215967 29245 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:40559
I20260811 03:15:26.216004 29325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:40559 every 8 connection(s)
I20260811 03:15:26.217661 29245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:15:26.220606 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 29245
I20260811 03:15:26.221505 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:44885
--local_ip_for_outbound_sockets=127.11.60.1
--tserver_master_addrs=127.11.60.62:40559
--webserver_port=44257
--webserver_interface=127.11.60.1
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:15:26.225415 29326 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:15:26.229014 29326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Bootstrap starting.
I20260811 03:15:26.250077 29326 log.cc:824] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Log is configured to *not* fsync() on all Append() calls
I20260811 03:15:26.262799 29326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:15:26.263367 29326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Bootstrap complete.
I20260811 03:15:26.272845 29326 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:26.273736 29326 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc513fcd6bde4c82a617037dc589ad95, State: Initialized, Role: FOLLOWER
I20260811 03:15:26.274464 29326 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:26.274734 29326 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:15:26.274883 29326 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:15:26.275065 29326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 1 FOLLOWER]: Advancing to term 2
I20260811 03:15:26.279541 29326 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:26.280103 29326 leader_election.cc:302] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc513fcd6bde4c82a617037dc589ad95; no voters:
I20260811 03:15:26.280782 29326 leader_election.cc:288] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 03:15:26.280936 29330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 03:15:26.282557 29330 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 2 LEADER]: Becoming Leader. State: Replica: bc513fcd6bde4c82a617037dc589ad95, State: Running, Role: LEADER
I20260811 03:15:26.283140 29330 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:26.283411 29326 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:15:26.285853 29332 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc513fcd6bde4c82a617037dc589ad95. Latest consensus state: current_term: 2 leader_uuid: "bc513fcd6bde4c82a617037dc589ad95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } } }
I20260811 03:15:26.286324 29332 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: This master's current role is: LEADER
I20260811 03:15:26.286979 29331 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bc513fcd6bde4c82a617037dc589ad95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } } }
I20260811 03:15:26.287346 29331 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: This master's current role is: LEADER
I20260811 03:15:26.290917 29338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:15:26.299563 29338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:15:26.301126 29338 catalog_manager.cc:1259] Loaded cluster ID: 427e50831f6549ef86f1aed7575a85af
I20260811 03:15:26.301278 29338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:15:26.305620 29338 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:15:26.307278 29338 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Loaded TSK: 0
I20260811 03:15:26.331689 29352 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 03:15:26.340939 29338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Generated new TSK 1
I20260811 03:15:26.341398 29338 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 03:15:26.588792 29328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:26.589159 29328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:26.589325 29328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:26.596856 29328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:26.596964 29328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:26.597136 29328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:15:26.607117 29328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:44885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=44257
--tserver_master_addrs=127.11.60.62:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:26.608469 29328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:26.609866 29328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:26.625039 29361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:26.625036 29360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:26.625710 29363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:26.626979 29328 server_base.cc:1034] running on GCE node
I20260811 03:15:26.627753 29328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:26.628988 29328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:26.630244 29328 hybrid_clock.cc:648] HybridClock initialized: now 1786418126630176 us; error 60 us; skew 500 ppm
I20260811 03:15:26.630651 29328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:26 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418126, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:26.634637 29328 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:15:26.636178 29328 webserver.cc:460] Webserver started at http://127.11.60.1:44257/ using document root <none> and password file <none>
I20260811 03:15:26.636902 29328 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:26.637064 29328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:26.643065 29328 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 03:15:26.646106 29372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:26.647707 29328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:15:26.647938 29328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "a5aa310eaad0427484df888834585d8b"
format_stamp: "Formatted at 2026-08-11 03:15:03 on dist-test-slave-tm1g"
server_key: "061ef81b9098058979b2fd5015e2e19a"
server_key_iv: "42a637313d6fd381aad82b86aa3ff868"
server_key_version: "encryptionkey@0"
I20260811 03:15:26.648947 29328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:26.668768 29328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:26.673154 29328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:26.673452 29328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:26.674583 29328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:15:26.675971 29328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:15:26.676102 29328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:26.676236 29328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:15:26.676352 29328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:26.711426 29328 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:44885
I20260811 03:15:26.711570 29500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:44885 every 8 connection(s)
I20260811 03:15:26.712829 29328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:15:26.713991 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 29328
I20260811 03:15:26.714905 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:38241
--local_ip_for_outbound_sockets=127.11.60.2
--tserver_master_addrs=127.11.60.62:40559
--webserver_port=41803
--webserver_interface=127.11.60.2
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:15:26 dist-test-slave-tm1g krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418126, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:15:26.739259 29503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:26.716368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:32895 (local address 127.11.60.62:40559)
0811 03:15:26.717041 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:26.717058 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:26.717678 (+ 620us) server_negotiation.cc:407] Connection header received
0811 03:15:26.719929 (+ 2251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:26.719950 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:26.720396 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:26.720796 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:26.722192 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:26.724308 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:26.725942 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:26.726769 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:26.726923 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:26.731564 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:26.731573 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:26.731587 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:26.731664 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:26.734668 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:26.735633 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:26.735640 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:26.735645 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:26.735806 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:26.736308 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:26.736311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:26.736313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:26.736799 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:26.736954 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:26.737934 (+ 980us) server_negotiation.cc:299] Negotiation successful
0811 03:15:26.738239 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":267,"threads_started":1}
I20260811 03:15:26.739905 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:26.716551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:32895)
0811 03:15:26.717478 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:26.717535 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:26.719362 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:26.720902 (+ 1540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:26.720910 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:26.721428 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:15:26.721980 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:26.721997 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:26.724508 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:26.724512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:26.725784 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:26.725796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:26.727032 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:26.727042 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:26.727353 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:26.728135 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:15:26.728171 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:15:26.731215 (+ 3044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:15:26.734814 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:26.734826 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:26.734844 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:26.735490 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:26.735937 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:26.735945 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:26.735951 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:26.736203 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:26.736983 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:15:26.736996 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:15:26.737448 (+ 452us) client_negotiation.cc:769] Sending connection context
0811 03:15:26.737762 (+ 314us) client_negotiation.cc:240] Negotiation successful
0811 03:15:26.738407 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":405,"threads_started":1}
I20260811 03:15:26.742970 29501 heartbeater.cc:347] Connected to a master server at 127.11.60.62:40559
I20260811 03:15:26.743301 29501 heartbeater.cc:464] Registering TS with master...
I20260811 03:15:26.744047 29501 heartbeater.cc:511] Master 127.11.60.62:40559 requested a full tablet report, sending...
I20260811 03:15:26.746270 29284 ts_manager.cc:194] Registered new tserver with Master: a5aa310eaad0427484df888834585d8b (127.11.60.1:44885)
I20260811 03:15:26.747828 29284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:32895
W20260811 03:15:27.064706 29505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:27.064983 29505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:27.065131 29505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:27.072324 29505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:27.072499 29505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:27.072726 29505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:15:27.083155 29505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:38241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=41803
--tserver_master_addrs=127.11.60.62:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:27.084618 29505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:27.085994 29505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:27.099576 29512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:27.099543 29513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:27.099498 29515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:27.100895 29505 server_base.cc:1034] running on GCE node
I20260811 03:15:27.101480 29505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:27.102474 29505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:27.103685 29505 hybrid_clock.cc:648] HybridClock initialized: now 1786418127103624 us; error 54 us; skew 500 ppm
I20260811 03:15:27.103973 29505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:27 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418127, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:27.107147 29505 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:15:27.108644 29505 webserver.cc:460] Webserver started at http://127.11.60.2:41803/ using document root <none> and password file <none>
I20260811 03:15:27.109338 29505 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:27.109458 29505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:27.115463 29505 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 03:15:27.118453 29523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:27.120054 29505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:15:27.120226 29505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "0ec3665007a342409d413d1ae84d7e5d"
format_stamp: "Formatted at 2026-08-11 03:15:04 on dist-test-slave-tm1g"
server_key: "ccd7859c07cf0f964e997259c48c9977"
server_key_iv: "847eb9a85670cd0c58b52bb7df693c1f"
server_key_version: "encryptionkey@0"
I20260811 03:15:27.121284 29505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:27.150475 29505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:27.155526 29505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:27.155923 29505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:27.157012 29505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:15:27.158548 29505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:15:27.158672 29505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:27.158802 29505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:15:27.158902 29505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:27.194348 29505 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:38241
I20260811 03:15:27.194407 29653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:38241 every 8 connection(s)
I20260811 03:15:27.195904 29505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:15:27.199492 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 29505
I20260811 03:15:27.200176 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:43271
--local_ip_for_outbound_sockets=127.11.60.3
--tserver_master_addrs=127.11.60.62:40559
--webserver_port=45331
--webserver_interface=127.11.60.3
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:15:27 dist-test-slave-tm1g krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418127, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:15:27.216473 29503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.198536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:39655 (local address 127.11.60.62:40559)
0811 03:15:27.198750 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.198756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.200396 (+ 1640us) server_negotiation.cc:407] Connection header received
0811 03:15:27.202291 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.202296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.202425 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.202540 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:27.204024 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.204741 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.206334 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.206777 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.206915 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.210460 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:27.210467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:27.210470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:27.210504 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:27.213226 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:27.213988 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:27.213993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:27.213995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:27.214112 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:27.214612 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:27.214618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:27.214621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:27.214913 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:27.215099 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.215847 (+ 748us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.216097 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 03:15:27.217453 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.199074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:39655)
0811 03:15:27.200230 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:27.200285 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:27.202017 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:27.202783 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:27.202791 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:27.203298 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:15:27.203841 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.203858 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.204913 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.204917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.206153 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.206166 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.207041 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.207049 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.207284 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.207976 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:15:27.208002 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:15:27.210188 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:15:27.213375 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:27.213384 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:27.213401 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:27.213843 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:27.214226 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:27.214231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:27.214235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:27.214495 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:27.215044 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:15:27.215052 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:15:27.215453 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 03:15:27.215747 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 03:15:27.216304 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"thread_start_us":358,"threads_started":1}
I20260811 03:15:27.219190 29654 heartbeater.cc:347] Connected to a master server at 127.11.60.62:40559
I20260811 03:15:27.219507 29654 heartbeater.cc:464] Registering TS with master...
I20260811 03:15:27.220247 29654 heartbeater.cc:511] Master 127.11.60.62:40559 requested a full tablet report, sending...
I20260811 03:15:27.221710 29284 ts_manager.cc:194] Registered new tserver with Master: 0ec3665007a342409d413d1ae84d7e5d (127.11.60.2:38241)
I20260811 03:15:27.222662 29284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:39655
W20260811 03:15:27.558949 29657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:27.559250 29657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:27.559446 29657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:27.567889 29657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:27.568022 29657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:27.568223 29657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:15:27.579674 29657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:43271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=45331
--tserver_master_addrs=127.11.60.62:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:27.581173 29657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:27.582757 29657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:27.598022 29663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:27.598053 29664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:27.598054 29667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:27.598798 29657 server_base.cc:1034] running on GCE node
I20260811 03:15:27.599465 29657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:27.600482 29657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:27.601701 29657 hybrid_clock.cc:648] HybridClock initialized: now 1786418127601658 us; error 31 us; skew 500 ppm
I20260811 03:15:27.602082 29657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:27 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418127, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:27.605597 29657 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:15:27.607206 29657 webserver.cc:460] Webserver started at http://127.11.60.3:45331/ using document root <none> and password file <none>
I20260811 03:15:27.607935 29657 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:27.608098 29657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:27.613987 29657 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 03:15:27.616670 29674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:27.618121 29657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 03:15:27.618294 29657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "b6d4833e8a5f4f55b31791ffe41dab2b"
format_stamp: "Formatted at 2026-08-11 03:15:04 on dist-test-slave-tm1g"
server_key: "974b19d296a9889b121736b39e11faae"
server_key_iv: "8343c29726352ca75253fab07c83285b"
server_key_version: "encryptionkey@0"
I20260811 03:15:27.619391 29657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:27.630031 29657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:27.634064 29657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:27.634414 29657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:27.635411 29657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:15:27.636755 29657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:15:27.636888 29657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:27.637012 29657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:15:27.637104 29657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:27.674079 29657 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:43271
I20260811 03:15:27.674216 29801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:43271 every 8 connection(s)
I20260811 03:15:27.675683 29657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:15:27.684942 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 29657
Aug 11 03:15:27 dist-test-slave-tm1g krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418127, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:15:27.698753 29503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.678684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:59805 (local address 127.11.60.62:40559)
0811 03:15:27.678896 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.678901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.680125 (+ 1224us) server_negotiation.cc:407] Connection header received
0811 03:15:27.681937 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.681941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.682011 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.682107 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:27.683664 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.684726 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.687028 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.687529 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.687670 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.691231 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:27.691236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:27.691240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:27.691275 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:27.694997 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:27.695983 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:27.695987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:27.695990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:27.696130 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:27.696689 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:27.696693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:27.696695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:27.697009 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:27.697133 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.698173 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.698485 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 03:15:27.700042 29804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.679211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:59805)
0811 03:15:27.679957 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:27.679997 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:27.681647 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:27.682369 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:27.682386 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:27.682917 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:15:27.683484 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.683501 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.684899 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.684906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.686860 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.686873 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.687767 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.687775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.688040 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.688755 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:15:27.688778 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:15:27.691013 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:15:27.695151 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:27.695159 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:27.695173 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:27.695811 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:27.696256 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:15:27.696261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:15:27.696266 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:15:27.696552 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:15:27.697157 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:15:27.697167 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:15:27.697696 (+ 529us) client_negotiation.cc:769] Sending connection context
0811 03:15:27.698048 (+ 352us) client_negotiation.cc:240] Negotiation successful
0811 03:15:27.698673 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":291,"threads_started":1}
I20260811 03:15:27.701747 29802 heartbeater.cc:347] Connected to a master server at 127.11.60.62:40559
I20260811 03:15:27.701942 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.685987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57140 (local address 127.11.60.62:40559)
0811 03:15:27.686557 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.686564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.686871 (+ 307us) server_negotiation.cc:407] Connection header received
0811 03:15:27.687110 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.687118 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.687187 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.687304 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:27.688989 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.689745 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.691264 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.691740 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.691902 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.695570 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:27.695578 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:27.695584 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:27.695626 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:27.699035 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:27.699771 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:27.699776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:27.699779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:27.699926 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:27.700374 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:27.700379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:27.700390 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:27.700732 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:27.701042 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.701390 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.701564 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":258,"threads_started":1}
I20260811 03:15:27.702428 29802 heartbeater.cc:464] Registering TS with master...
I20260811 03:15:27.703178 29802 heartbeater.cc:511] Master 127.11.60.62:40559 requested a full tablet report, sending...
I20260811 03:15:27.704923 29284 ts_manager.cc:194] Registered new tserver with Master: b6d4833e8a5f4f55b31791ffe41dab2b (127.11.60.3:43271)
I20260811 03:15:27.705788 29284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:59805
I20260811 03:15:27.717262 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:15:27.724512 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.718467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57146 (local address 127.11.60.62:40559)
0811 03:15:27.718651 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.718655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.719129 (+ 474us) server_negotiation.cc:407] Connection header received
0811 03:15:27.719285 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.719289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.719493 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.719629 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:15:27.720065 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.720835 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.722454 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.722933 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.723040 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.723582 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:15:27.723884 (+ 302us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 03:15:27.724200 (+ 316us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57146: token expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 03:15:27.724673 29806 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57146: token expired
W20260811 03:15:27.725327 29196 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.60.62:40559)
I20260811 03:15:27.739040 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.725895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57154 (local address 127.11.60.62:40559)
0811 03:15:27.726103 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.726110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.726129 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:15:27.726356 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.726361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.726434 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.726538 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:27.728062 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.728818 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.730183 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.730707 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.730845 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.733408 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:27.733414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:27.733418 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:27.733461 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:27.736173 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:27.736912 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:27.736919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:27.736923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:27.737083 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:27.737472 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:27.737478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:27.737482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:27.737833 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:27.738088 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.738466 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.738703 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 03:15:27.750876 29501 heartbeater.cc:503] Master 127.11.60.62:40559 was elected leader, sending a full tablet report...
I20260811 03:15:27.743871 29284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:15:27.754798 29284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:15:27.790975 29814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.767670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:44885 (local address 127.0.0.1:33796)
0811 03:15:27.768549 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:27.768587 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:27.768749 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:27.784191 (+ 15442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:27.784199 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:27.784230 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:15:27.784633 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.784646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.786210 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.786215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.788309 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.788325 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.789808 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.789814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.790025 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.790090 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 03:15:27.790244 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 03:15:27.790491 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":95,"thread_start_us":475,"threads_started":1}
I20260811 03:15:27.791651 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.767811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:43271 (local address 127.0.0.1:46050)
0811 03:15:27.780639 (+ 12828us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:27.780666 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:27.780862 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:27.782086 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:27.782100 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:27.782149 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:15:27.782736 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.782750 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.784973 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.784979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.788031 (+ 3052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.788048 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.790183 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.790191 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.790402 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.790436 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 03:15:27.790509 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 03:15:27.791201 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12646,"mutex_wait_us":10,"thread_start_us":13431,"threads_started":1}
I20260811 03:15:27.791951 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.768711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.11.60.3:43271)
0811 03:15:27.777380 (+ 8669us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.777393 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.781034 (+ 3641us) server_negotiation.cc:407] Connection header received
0811 03:15:27.781172 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.781184 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.781500 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.781733 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:15:27.783016 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.784808 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.788222 (+ 3414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.789716 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.789851 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.790083 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.790900 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.791367 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8417,"thread_start_us":8189,"threads_started":1}
I20260811 03:15:27.791954 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.767883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33796 (local address 127.11.60.1:44885)
0811 03:15:27.783554 (+ 15671us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.783566 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.783596 (+ 30us) server_negotiation.cc:407] Connection header received
0811 03:15:27.783700 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.783707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.783978 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.784174 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:15:27.784848 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.786044 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.788489 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.789566 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.789731 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.790003 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.790349 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.791508 (+ 1159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15479,"thread_start_us":15291,"threads_started":1}
I20260811 03:15:27.793473 29819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.769328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45894 (local address 127.11.60.2:38241)
0811 03:15:27.782780 (+ 13452us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.782789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.782818 (+ 29us) server_negotiation.cc:407] Connection header received
0811 03:15:27.783161 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.783174 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.783467 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.783673 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:15:27.784309 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.785971 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.788717 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.790194 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.792010 (+ 1816us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.792239 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.792867 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.793116 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13270,"thread_start_us":13125,"threads_started":1}
I20260811 03:15:27.794373 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.768939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:38241 (local address 127.0.0.1:45894)
0811 03:15:27.781527 (+ 12588us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:27.781548 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:27.781671 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:27.783658 (+ 1987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:27.783663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:27.783689 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:15:27.784089 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.784100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.786151 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.786157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.788538 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.788553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.791657 (+ 3104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.791663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.792642 (+ 979us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.792678 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 03:15:27.792785 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 03:15:27.793912 (+ 1127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12452,"mutex_wait_us":103,"thread_start_us":13766,"threads_started":1}
I20260811 03:15:27.795814 29426 tablet_service.cc:1467] Processing CreateTablet for tablet 582d011b4c9a4e639b3c6cc99b2d83e0 (DEFAULT_TABLE table=test-table [id=9646329b81bf477eaa9e76d7a17088c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:15:27.795818 29730 tablet_service.cc:1467] Processing CreateTablet for tablet 582d011b4c9a4e639b3c6cc99b2d83e0 (DEFAULT_TABLE table=test-table [id=9646329b81bf477eaa9e76d7a17088c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:15:27.797173 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 582d011b4c9a4e639b3c6cc99b2d83e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:15:27.797209 29730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 582d011b4c9a4e639b3c6cc99b2d83e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:15:27.798041 29582 tablet_service.cc:1467] Processing CreateTablet for tablet 582d011b4c9a4e639b3c6cc99b2d83e0 (DEFAULT_TABLE table=test-table [id=9646329b81bf477eaa9e76d7a17088c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:15:27.799424 29582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 582d011b4c9a4e639b3c6cc99b2d83e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:15:27.810456 29820 tablet_bootstrap.cc:492] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: Bootstrap starting.
I20260811 03:15:27.813584 29821 tablet_bootstrap.cc:492] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: Bootstrap starting.
I20260811 03:15:27.814445 29820 tablet_bootstrap.cc:654] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: Neither blocks nor log segments found. Creating new log.
I20260811 03:15:27.815467 29822 tablet_bootstrap.cc:492] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: Bootstrap starting.
I20260811 03:15:27.817020 29820 log.cc:824] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: Log is configured to *not* fsync() on all Append() calls
I20260811 03:15:27.817430 29821 tablet_bootstrap.cc:654] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: Neither blocks nor log segments found. Creating new log.
I20260811 03:15:27.818719 29821 log.cc:824] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: Log is configured to *not* fsync() on all Append() calls
I20260811 03:15:27.819442 29822 tablet_bootstrap.cc:654] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: Neither blocks nor log segments found. Creating new log.
I20260811 03:15:27.820608 29822 log.cc:824] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: Log is configured to *not* fsync() on all Append() calls
I20260811 03:15:27.820798 29820 tablet_bootstrap.cc:492] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: No bootstrap required, opened a new log
I20260811 03:15:27.821100 29820 ts_tablet_manager.cc:1397] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260811 03:15:27.821954 29821 tablet_bootstrap.cc:492] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: No bootstrap required, opened a new log
I20260811 03:15:27.822233 29821 ts_tablet_manager.cc:1397] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 03:15:27.823513 29822 tablet_bootstrap.cc:492] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: No bootstrap required, opened a new log
I20260811 03:15:27.823782 29822 ts_tablet_manager.cc:1397] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 03:15:27.831666 29822 raft_consensus.cc:348] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:27.832065 29822 raft_consensus.cc:374] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:15:27.832157 29822 raft_consensus.cc:729] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec3665007a342409d413d1ae84d7e5d, State: Initialized, Role: FOLLOWER
I20260811 03:15:27.832814 29822 consensus_queue.cc:260] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:27.832787 29820 raft_consensus.cc:348] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:27.832845 29821 raft_consensus.cc:348] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:27.833250 29821 raft_consensus.cc:374] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:15:27.833251 29820 raft_consensus.cc:374] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:15:27.833365 29821 raft_consensus.cc:729] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d4833e8a5f4f55b31791ffe41dab2b, State: Initialized, Role: FOLLOWER
I20260811 03:15:27.833365 29820 raft_consensus.cc:729] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5aa310eaad0427484df888834585d8b, State: Initialized, Role: FOLLOWER
I20260811 03:15:27.834139 29654 heartbeater.cc:503] Master 127.11.60.62:40559 was elected leader, sending a full tablet report...
I20260811 03:15:27.834060 29821 consensus_queue.cc:260] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:27.834880 29822 ts_tablet_manager.cc:1428] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: Time spent starting tablet: real 0.011s user 0.003s sys 0.009s
I20260811 03:15:27.835270 29802 heartbeater.cc:503] Master 127.11.60.62:40559 was elected leader, sending a full tablet report...
I20260811 03:15:27.835331 29820 consensus_queue.cc:260] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:27.836061 29821 ts_tablet_manager.cc:1428] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260811 03:15:27.839325 29820 ts_tablet_manager.cc:1428] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: Time spent starting tablet: real 0.018s user 0.016s sys 0.002s
I20260811 03:15:27.864125 29826 raft_consensus.cc:482] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:15:27.864439 29826 raft_consensus.cc:504] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:27.866346 29826 leader_election.cc:288] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5aa310eaad0427484df888834585d8b (127.11.60.1:44885), b6d4833e8a5f4f55b31791ffe41dab2b (127.11.60.3:43271)
I20260811 03:15:27.876874 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.867022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:55837 (local address 127.11.60.3:43271)
0811 03:15:27.867271 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.867280 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.868338 (+ 1058us) server_negotiation.cc:407] Connection header received
0811 03:15:27.868661 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.868665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.868730 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.868810 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:15:27.869628 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.870827 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.874182 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.875540 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.875691 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.875806 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.876478 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.876624 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:15:27.876901 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.866954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:47373 (local address 127.11.60.1:44885)
0811 03:15:27.867212 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.867222 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.868028 (+ 806us) server_negotiation.cc:407] Connection header received
0811 03:15:27.868360 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.868368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.868488 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.868563 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:15:27.869636 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.870827 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.874182 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.875535 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.875691 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.875806 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.876479 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.876624 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 03:15:27.877134 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.866519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:44885 (local address 127.11.60.2:47373)
0811 03:15:27.867904 (+ 1385us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:27.867931 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:27.868162 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:27.868757 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:27.868760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:27.868807 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:15:27.869475 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.869488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.871048 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.871052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.873964 (+ 2912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.873980 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.875854 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.875859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.876145 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.876230 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 03:15:27.876431 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 03:15:27.876551 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1216,"mutex_wait_us":298,"spinlock_wait_cycles":25344,"thread_start_us":1003,"threads_started":1}
I20260811 03:15:27.877243 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.866853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:43271 (local address 127.11.60.2:55837)
0811 03:15:27.868241 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:27.868259 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:15:27.868449 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:15:27.868856 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:15:27.868862 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:15:27.868885 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:15:27.869399 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.869416 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.871046 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.871052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.873963 (+ 2911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:15:27.873980 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.875844 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.875852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:15:27.876145 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.876230 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 03:15:27.876340 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 03:15:27.876491 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1246,"mutex_wait_us":259,"thread_start_us":991,"threads_started":1}
I20260811 03:15:27.878264 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582d011b4c9a4e639b3c6cc99b2d83e0" candidate_uuid: "0ec3665007a342409d413d1ae84d7e5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5aa310eaad0427484df888834585d8b" is_pre_election: true
I20260811 03:15:27.878398 29751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582d011b4c9a4e639b3c6cc99b2d83e0" candidate_uuid: "0ec3665007a342409d413d1ae84d7e5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" is_pre_election: true
I20260811 03:15:27.878890 29751 raft_consensus.cc:2454] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec3665007a342409d413d1ae84d7e5d in term 0.
I20260811 03:15:27.878894 29446 raft_consensus.cc:2454] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec3665007a342409d413d1ae84d7e5d in term 0.
I20260811 03:15:27.879637 29525 leader_election.cc:302] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0ec3665007a342409d413d1ae84d7e5d, a5aa310eaad0427484df888834585d8b, b6d4833e8a5f4f55b31791ffe41dab2b; no voters:
I20260811 03:15:27.880117 29826 raft_consensus.cc:2781] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:15:27.880285 29826 raft_consensus.cc:482] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:15:27.880404 29826 raft_consensus.cc:3037] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:15:27.882690 29826 raft_consensus.cc:504] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:27.883337 29826 leader_election.cc:288] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [CANDIDATE]: Term 1 election: Requested vote from peers a5aa310eaad0427484df888834585d8b (127.11.60.1:44885), b6d4833e8a5f4f55b31791ffe41dab2b (127.11.60.3:43271)
I20260811 03:15:27.883805 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582d011b4c9a4e639b3c6cc99b2d83e0" candidate_uuid: "0ec3665007a342409d413d1ae84d7e5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5aa310eaad0427484df888834585d8b"
I20260811 03:15:27.884001 29751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582d011b4c9a4e639b3c6cc99b2d83e0" candidate_uuid: "0ec3665007a342409d413d1ae84d7e5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b"
I20260811 03:15:27.884073 29446 raft_consensus.cc:3037] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:15:27.884254 29751 raft_consensus.cc:3037] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:15:27.886521 29446 raft_consensus.cc:2454] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec3665007a342409d413d1ae84d7e5d in term 1.
I20260811 03:15:27.886754 29751 raft_consensus.cc:2454] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec3665007a342409d413d1ae84d7e5d in term 1.
I20260811 03:15:27.886988 29525 leader_election.cc:302] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec3665007a342409d413d1ae84d7e5d, a5aa310eaad0427484df888834585d8b; no voters:
I20260811 03:15:27.887326 29826 raft_consensus.cc:2781] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:15:27.887856 29826 raft_consensus.cc:686] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 1 LEADER]: Becoming Leader. State: Replica: 0ec3665007a342409d413d1ae84d7e5d, State: Running, Role: LEADER
I20260811 03:15:27.888401 29826 consensus_queue.cc:237] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } }
I20260811 03:15:27.893033 29283 catalog_manager.cc:5495] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec3665007a342409d413d1ae84d7e5d (127.11.60.2). New cstate: current_term: 1 leader_uuid: "0ec3665007a342409d413d1ae84d7e5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec3665007a342409d413d1ae84d7e5d" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 38241 } health_report { overall_health: HEALTHY } } }
I20260811 03:15:27.925319 29819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.919840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45906 (local address 127.11.60.2:38241)
0811 03:15:27.920028 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.920033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.920051 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:15:27.920140 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.920144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.920231 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.920367 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:15:27.920747 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.921487 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.923018 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.923506 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.923607 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.924135 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:15:27.924647 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:15:27.924759 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.924930 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.925056 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 03:15:27.927516 29803 tablet.cc:2369] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:15:27.933993 29751 raft_consensus.cc:1261] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b [term 1 FOLLOWER]: Refusing update from remote peer 0ec3665007a342409d413d1ae84d7e5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:15:27.933997 29446 raft_consensus.cc:1261] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b [term 1 FOLLOWER]: Refusing update from remote peer 0ec3665007a342409d413d1ae84d7e5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:15:27.935122 29826 consensus_queue.cc:1035] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5aa310eaad0427484df888834585d8b" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:15:27.935456 29831 consensus_queue.cc:1035] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6d4833e8a5f4f55b31791ffe41dab2b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:15:27.947698 29835 mvcc.cc:205] Tried to move back new op lower bound from 7317168652007743488 to 7317168651834871808. Current Snapshot: MvccSnapshot[applied={T|T < 7317168652007743488}]
I20260811 03:15:27.950224 29836 mvcc.cc:205] Tried to move back new op lower bound from 7317168652007743488 to 7317168651834871808. Current Snapshot: MvccSnapshot[applied={T|T < 7317168652007743488}]
I20260811 03:15:27.962137 29282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:15:27.962515 29282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:15:27.966318 29282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Sending DeleteTablet for 3 replicas of tablet 582d011b4c9a4e639b3c6cc99b2d83e0
I20260811 03:15:27.967564 29730 tablet_service.cc:1514] Processing DeleteTablet for tablet 582d011b4c9a4e639b3c6cc99b2d83e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:15:27 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:46050
I20260811 03:15:27.967671 29426 tablet_service.cc:1514] Processing DeleteTablet for tablet 582d011b4c9a4e639b3c6cc99b2d83e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:15:27 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:33796
I20260811 03:15:27.967792 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 29328
I20260811 03:15:27.968740 29847 tablet_replica.cc:330] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: stopping tablet replica
I20260811 03:15:27.968816 29582 tablet_service.cc:1514] Processing DeleteTablet for tablet 582d011b4c9a4e639b3c6cc99b2d83e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:15:27 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:45894
I20260811 03:15:27.969175 29847 raft_consensus.cc:2229] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:15:27.969648 29847 raft_consensus.cc:2258] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:15:27.970521 29846 tablet_replica.cc:330] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: stopping tablet replica
I20260811 03:15:27.970932 29846 raft_consensus.cc:2229] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:15:27.971144 29848 tablet_replica.cc:330] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: stopping tablet replica
I20260811 03:15:27.971513 29848 raft_consensus.cc:2229] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:15:27.972235 29848 raft_consensus.cc:2258] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:15:27.972534 29847 ts_tablet_manager.cc:1905] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:15:27.973212 29846 raft_consensus.cc:2258] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:15:27.975301 29846 ts_tablet_manager.cc:1905] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:15:27.975656 29848 ts_tablet_manager.cc:1905] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:15:27.980211 29847 ts_tablet_manager.cc:1918] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:15:27.980386 29847 log.cc:1196] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/582d011b4c9a4e639b3c6cc99b2d83e0
I20260811 03:15:27.980980 29847 ts_tablet_manager.cc:1939] T 582d011b4c9a4e639b3c6cc99b2d83e0 P a5aa310eaad0427484df888834585d8b: Deleting consensus metadata
I20260811 03:15:27.982658 29848 ts_tablet_manager.cc:1918] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:15:27.982847 29848 log.cc:1196] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/582d011b4c9a4e639b3c6cc99b2d83e0
I20260811 03:15:27.983093 29265 catalog_manager.cc:4841] TS a5aa310eaad0427484df888834585d8b (127.11.60.1:44885): tablet 582d011b4c9a4e639b3c6cc99b2d83e0 (table test-table [id=9646329b81bf477eaa9e76d7a17088c0]) successfully deleted
I20260811 03:15:27.983402 29848 ts_tablet_manager.cc:1939] T 582d011b4c9a4e639b3c6cc99b2d83e0 P 0ec3665007a342409d413d1ae84d7e5d: Deleting consensus metadata
I20260811 03:15:27.983399 29846 ts_tablet_manager.cc:1918] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:15:27.983671 29846 log.cc:1196] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/582d011b4c9a4e639b3c6cc99b2d83e0
I20260811 03:15:27.984357 29846 ts_tablet_manager.cc:1939] T 582d011b4c9a4e639b3c6cc99b2d83e0 P b6d4833e8a5f4f55b31791ffe41dab2b: Deleting consensus metadata
I20260811 03:15:27.985162 29265 catalog_manager.cc:4841] TS 0ec3665007a342409d413d1ae84d7e5d (127.11.60.2:38241): tablet 582d011b4c9a4e639b3c6cc99b2d83e0 (table test-table [id=9646329b81bf477eaa9e76d7a17088c0]) successfully deleted
I20260811 03:15:27.986203 29266 catalog_manager.cc:4841] TS b6d4833e8a5f4f55b31791ffe41dab2b (127.11.60.3:43271): tablet 582d011b4c9a4e639b3c6cc99b2d83e0 (table test-table [id=9646329b81bf477eaa9e76d7a17088c0]) successfully deleted
I20260811 03:15:27.989578 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:27.970297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.11.60.1:44885)
0811 03:15:27.970508 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:27.970525 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:27.970550 (+ 25us) server_negotiation.cc:407] Connection header received
0811 03:15:27.970609 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:27.970615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:27.970701 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:27.970813 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:27.972912 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.974115 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.976378 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:27.977072 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:27.977195 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:27.980951 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:27.980971 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:27.980989 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:27.981041 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:27.986586 (+ 5545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:27.987378 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:27.987385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:27.987387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:27.987519 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:27.987948 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:27.987951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:27.987953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:27.988319 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:27.988641 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:27.989000 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 03:15:27.989233 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 03:15:27.990207 29496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:28.103502 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 29328
I20260811 03:15:28.126019 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 29505
I20260811 03:15:28.142670 29819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:28.126891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45914 (local address 127.11.60.2:38241)
0811 03:15:28.127130 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:28.127137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:28.127211 (+ 74us) server_negotiation.cc:407] Connection header received
0811 03:15:28.127454 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:28.127459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:28.127553 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:28.127676 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:28.129089 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:28.129948 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:28.131514 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:28.131976 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:28.132129 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:28.134924 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:28.134939 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:28.134956 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:28.134996 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:28.138764 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:28.139776 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:28.139787 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:28.139791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:28.139991 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:28.140597 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:28.140602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:28.140606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:28.141261 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:28.141607 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:28.141990 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 03:15:28.142227 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 03:15:28.143500 29649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:28.260823 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 29505
I20260811 03:15:28.279976 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 29657
I20260811 03:15:28.298734 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:28.280777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.11.60.3:43271)
0811 03:15:28.280997 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 03:15:28.281005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:15:28.281130 (+ 125us) server_negotiation.cc:407] Connection header received
0811 03:15:28.281382 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:15:28.281387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:15:28.281454 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:15:28.281571 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:15:28.283306 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:28.284136 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:28.285498 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:15:28.285904 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:15:28.286049 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:15:28.292486 (+ 6437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:15:28.292504 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:15:28.292521 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:15:28.292566 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:15:28.295804 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:28.296571 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:28.296578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:28.296580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:28.296732 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:15:28.297144 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:15:28.297147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:15:28.297149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:15:28.297514 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:15:28.297735 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 03:15:28.298094 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 03:15:28.298279 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 03:15:28.299548 29796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:28.392402 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 29657
I20260811 03:15:28.412428 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 29245
I20260811 03:15:28.413782 29321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:15:28.532977 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 29245
I20260811 03:15:48.559520 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40559
--webserver_interface=127.11.60.62
--webserver_port=34779
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:40559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:15:48.928911 29854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:48.929245 29854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:48.929360 29854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:48.937774 29854 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 03:15:48.937896 29854 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 03:15:48.937942 29854 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 03:15:48.937994 29854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:15:48.938055 29854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:48.938134 29854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:48.938187 29854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:15:48.938247 29854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:15:48.950070 29854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:40559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=34779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:48.951579 29854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:48.953169 29854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 03:15:48.966759 29854 server_base.cc:1034] running on GCE node
W20260811 03:15:48.966778 29859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:48.966753 29860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:48.966840 29863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:48.968037 29854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:48.969632 29854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:48.970912 29854 hybrid_clock.cc:648] HybridClock initialized: now 1786418148970839 us; error 48 us; skew 500 ppm
I20260811 03:15:48.971318 29854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:48 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418148, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:48.979840 29854 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:15:48.981551 29854 webserver.cc:460] Webserver started at http://127.11.60.62:34779/ using document root <none> and password file <none>
I20260811 03:15:48.982553 29854 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:48.982677 29854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:48.988706 29854 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 03:15:48.996098 29872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:48.997742 29854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:15:48.997967 29854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "bc513fcd6bde4c82a617037dc589ad95"
format_stamp: "Formatted at 2026-08-11 03:15:03 on dist-test-slave-tm1g"
server_key: "0be2d762f7247deb56c87cde34daa665"
server_key_iv: "29e3519c34728150c1481eee6f139628"
server_key_version: "encryptionkey@0"
I20260811 03:15:48.999109 29854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:49.017199 29854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:49.022432 29854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:49.022796 29854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:49.044859 29854 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:40559
I20260811 03:15:49.044895 29934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:40559 every 8 connection(s)
I20260811 03:15:49.046710 29854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:15:49.054527 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 29854
I20260811 03:15:49.054924 29935 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:15:49.055761 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:44885
--local_ip_for_outbound_sockets=127.11.60.1
--tserver_master_addrs=127.11.60.62:40559
--webserver_port=44257
--webserver_interface=127.11.60.1
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:15:49.060879 29935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Bootstrap starting.
I20260811 03:15:49.086189 29935 log.cc:824] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Log is configured to *not* fsync() on all Append() calls
I20260811 03:15:49.104480 29935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:15:49.105098 29935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Bootstrap complete.
I20260811 03:15:49.115144 29935 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:49.116081 29935 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc513fcd6bde4c82a617037dc589ad95, State: Initialized, Role: FOLLOWER
I20260811 03:15:49.116770 29935 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:49.117029 29935 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:15:49.117151 29935 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:15:49.117347 29935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 2 FOLLOWER]: Advancing to term 3
I20260811 03:15:49.121435 29935 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:49.121984 29935 leader_election.cc:302] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc513fcd6bde4c82a617037dc589ad95; no voters:
I20260811 03:15:49.122803 29935 leader_election.cc:288] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 03:15:49.122987 29942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 03:15:49.125379 29935 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:15:49.125351 29942 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [term 3 LEADER]: Becoming Leader. State: Replica: bc513fcd6bde4c82a617037dc589ad95, State: Running, Role: LEADER
I20260811 03:15:49.126173 29942 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } }
I20260811 03:15:49.129408 29944 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc513fcd6bde4c82a617037dc589ad95. Latest consensus state: current_term: 3 leader_uuid: "bc513fcd6bde4c82a617037dc589ad95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } } }
I20260811 03:15:49.129782 29944 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: This master's current role is: LEADER
I20260811 03:15:49.130419 29943 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bc513fcd6bde4c82a617037dc589ad95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc513fcd6bde4c82a617037dc589ad95" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40559 } } }
I20260811 03:15:49.130740 29943 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95 [sys.catalog]: This master's current role is: LEADER
I20260811 03:15:49.141697 29954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:15:49.150921 29954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:15:49.152745 29954 catalog_manager.cc:1259] Loaded cluster ID: 427e50831f6549ef86f1aed7575a85af
I20260811 03:15:49.152968 29954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:15:49.157271 29954 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:15:49.159132 29954 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Loaded TSK: 0
I20260811 03:15:49.159380 29954 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc513fcd6bde4c82a617037dc589ad95: Loaded TSK: 1
F20260811 03:15:49.160652 29954 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6b845969ad google::LogMessage::Fail() at ??:0
@ 0x7f6b8459988a google::LogMessage::SendToLog() at ??:0
@ 0x7f6b845964de google::LogMessage::Flush() at ??:0
@ 0x7f6b84597a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6b882fa7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6b882e973e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6b99fc51c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6b99fc4783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6b9a0502a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f6b9a0500c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f6b9a09a5e9 std::function<>::operator()() at ??:0
@ 0x7f6b99fc8cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f6b99fc6c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6b9a04e610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f6b9a04e439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6b871e2345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6b871e591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6b871e5799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6b871c0eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6b8a49a6db start_thread at ??:0
@ 0x7f6b8343061f clone at ??:0
W20260811 03:15:49.504104 29939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:49.504426 29939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:49.504528 29939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:49.511983 29939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:49.512099 29939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:49.512213 29939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:15:49.522322 29939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:44885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=44257
--tserver_master_addrs=127.11.60.62:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:49.523633 29939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:49.524919 29939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:49.538643 29993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:49.538658 29997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:49.538592 29994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:49.539688 29939 server_base.cc:1034] running on GCE node
I20260811 03:15:49.540472 29939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:49.541520 29939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:49.542752 29939 hybrid_clock.cc:648] HybridClock initialized: now 1786418149542701 us; error 45 us; skew 500 ppm
I20260811 03:15:49.543080 29939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:49 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418149, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:49.546221 29939 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:15:49.547627 29939 webserver.cc:460] Webserver started at http://127.11.60.1:44257/ using document root <none> and password file <none>
I20260811 03:15:49.548322 29939 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:49.548483 29939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:49.553804 29939 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:15:49.556586 30005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:49.557819 29939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:15:49.558040 29939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "a5aa310eaad0427484df888834585d8b"
format_stamp: "Formatted at 2026-08-11 03:15:03 on dist-test-slave-tm1g"
server_key: "061ef81b9098058979b2fd5015e2e19a"
server_key_iv: "42a637313d6fd381aad82b86aa3ff868"
server_key_version: "encryptionkey@0"
I20260811 03:15:49.559082 29939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:49.575228 29939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:49.579242 29939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:49.579530 29939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:49.580554 29939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:15:49.581923 29939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:15:49.582055 29939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:49.582190 29939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:15:49.582331 29939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:49.617359 29939 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:44885
I20260811 03:15:49.617450 30129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:44885 every 8 connection(s)
I20260811 03:15:49.618904 29939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:15:49.622377 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 29939
I20260811 03:15:49.623272 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:38241
--local_ip_for_outbound_sockets=127.11.60.2
--tserver_master_addrs=127.11.60.62:40559
--webserver_port=41803
--webserver_interface=127.11.60.2
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:15:49.624455 30132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:49.621998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:50769)
0811 03:15:49.622980 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:49.623267 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":407,"threads_started":1}
W20260811 03:15:49.625196 30130 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:40559 (0 consecutive failures): Network error: Failed to ping master at 127.11.60.62:40559: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
I20260811 03:15:49.626202 30132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:49.625830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:58975)
0811 03:15:49.626015 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:49.626089 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 03:15:49.627256 30132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:49.626913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:33071)
0811 03:15:49.627046 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:49.627123 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 03:15:49.627554 30130 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 03:15:49.991882 30133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:49.992231 30133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:49.992375 30133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:49.999876 30133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:50.000000 30133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:50.000115 30133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:15:50.009990 30133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:38241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=41803
--tserver_master_addrs=127.11.60.62:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:50.011395 30133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:50.012787 30133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:50.026399 30139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:50.026292 30142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:50.026371 30140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:50.027204 30133 server_base.cc:1034] running on GCE node
I20260811 03:15:50.027832 30133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:50.028746 30133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:50.029985 30133 hybrid_clock.cc:648] HybridClock initialized: now 1786418150029915 us; error 64 us; skew 500 ppm
I20260811 03:15:50.030364 30133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:50 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418150, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:50.033622 30133 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:15:50.035203 30133 webserver.cc:460] Webserver started at http://127.11.60.2:41803/ using document root <none> and password file <none>
I20260811 03:15:50.035986 30133 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:50.036140 30133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:50.041728 30133 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 03:15:50.044451 30151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:50.045794 30133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:15:50.045981 30133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "0ec3665007a342409d413d1ae84d7e5d"
format_stamp: "Formatted at 2026-08-11 03:15:04 on dist-test-slave-tm1g"
server_key: "ccd7859c07cf0f964e997259c48c9977"
server_key_iv: "847eb9a85670cd0c58b52bb7df693c1f"
server_key_version: "encryptionkey@0"
I20260811 03:15:50.047068 30133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:50.073133 30133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:50.077564 30133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:50.077867 30133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:50.079099 30133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:15:50.080528 30133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:15:50.080679 30133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:50.080806 30133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:15:50.080896 30133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:50.117228 30133 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:38241
I20260811 03:15:50.117318 30275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:38241 every 8 connection(s)
I20260811 03:15:50.118778 30133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:15:50.123847 30279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:50.121832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:37863)
0811 03:15:50.122685 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:50.122949 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":328,"threads_started":1}
W20260811 03:15:50.124626 30277 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:40559 (0 consecutive failures): Network error: Failed to ping master at 127.11.60.62:40559: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
I20260811 03:15:50.125073 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 30133
I20260811 03:15:50.125589 30279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:50.125217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:59689)
0811 03:15:50.125390 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:50.125466 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 03:15:50.125787 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:43271
--local_ip_for_outbound_sockets=127.11.60.3
--tserver_master_addrs=127.11.60.62:40559
--webserver_port=45331
--webserver_interface=127.11.60.3
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:15:50.126518 30279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:50.126143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:37921)
0811 03:15:50.126282 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:50.126408 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 03:15:50.126782 30277 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 03:15:50.478266 30280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:15:50.478600 30280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:15:50.478760 30280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:15:50.486183 30280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:15:50.486316 30280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:15:50.486502 30280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:15:50.496596 30280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:43271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=45331
--tserver_master_addrs=127.11.60.62:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:15:50.498072 30280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:15:50.499504 30280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:15:50.514082 30286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:50.514056 30290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:15:50.513983 30287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:15:50.514948 30280 server_base.cc:1034] running on GCE node
I20260811 03:15:50.515703 30280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:15:50.516812 30280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:15:50.518085 30280 hybrid_clock.cc:648] HybridClock initialized: now 1786418150518020 us; error 47 us; skew 500 ppm
I20260811 03:15:50.518532 30280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:15:50 dist-test-slave-tm1g krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418150, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:15:50.521957 30280 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:15:50.523600 30280 webserver.cc:460] Webserver started at http://127.11.60.3:45331/ using document root <none> and password file <none>
I20260811 03:15:50.524405 30280 fs_manager.cc:359] Metadata directory not provided
I20260811 03:15:50.524518 30280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:15:50.530567 30280 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 03:15:50.533412 30297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:50.535014 30280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:15:50.535208 30280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "b6d4833e8a5f4f55b31791ffe41dab2b"
format_stamp: "Formatted at 2026-08-11 03:15:04 on dist-test-slave-tm1g"
server_key: "974b19d296a9889b121736b39e11faae"
server_key_iv: "8343c29726352ca75253fab07c83285b"
server_key_version: "encryptionkey@0"
I20260811 03:15:50.536227 30280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:15:50.555464 30280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:15:50.559754 30280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:15:50.560038 30280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:15:50.561059 30280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:15:50.562552 30280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:15:50.562652 30280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:50.562812 30280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:15:50.562909 30280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:15:50.598433 30280 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:43271
I20260811 03:15:50.598616 30427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:43271 every 8 connection(s)
I20260811 03:15:50.600049 30280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:15:50.605098 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:50.603225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:57175)
0811 03:15:50.603995 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:50.604214 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":311,"threads_started":1}
W20260811 03:15:50.605804 30428 heartbeater.cc:644] Failed to heartbeat to 127.11.60.62:40559 (0 consecutive failures): Network error: Failed to ping master at 127.11.60.62:40559: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
I20260811 03:15:50.606776 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:50.606402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:60159)
0811 03:15:50.606587 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:50.606664 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 03:15:50.607767 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:50.607309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:43023)
0811 03:15:50.607474 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:50.607601 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 03:15:50.607867 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 30280
W20260811 03:15:50.608111 30428 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 03:15:50.628995 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:50.628122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:60021)
0811 03:15:50.628639 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:50.628753 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":245,"threads_started":1}
I20260811 03:15:51.128363 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:51.127503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:41069)
0811 03:15:51.128023 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:51.128115 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":261,"threads_started":1}
I20260811 03:15:51.609644 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:51.608739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:48123)
0811 03:15:51.609293 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:51.609407 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":260,"threads_started":1}
I20260811 03:15:51.630833 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:51.629989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:48025)
0811 03:15:51.630571 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:51.630656 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":326,"threads_started":1}
I20260811 03:15:52.130182 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:52.129416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:33373)
0811 03:15:52.129913 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:52.130006 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":251,"threads_started":1}
I20260811 03:15:52.611385 30438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:52.610618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:52861)
0811 03:15:52.611124 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:52.611208 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":265,"threads_started":1}
I20260811 03:15:52.632450 30439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:52.631689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:33795)
0811 03:15:52.632200 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:52.632280 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":287,"threads_started":1}
I20260811 03:15:53.131966 30440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:53.131169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:40823)
0811 03:15:53.131671 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:53.131759 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":279,"threads_started":1}
I20260811 03:15:53.613095 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:53.612364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:58461)
0811 03:15:53.612820 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:53.612906 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":230,"threads_started":1}
I20260811 03:15:53.634064 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:53.633365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:41767)
0811 03:15:53.633786 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:53.633867 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":201,"threads_started":1}
I20260811 03:15:54.133631 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:54.132888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:58371)
0811 03:15:54.133337 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:54.133451 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":239,"threads_started":1}
I20260811 03:15:54.614830 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:54.614047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:56849)
0811 03:15:54.614547 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:54.614631 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":327,"threads_started":1}
I20260811 03:15:54.635897 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:54.635063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:40511)
0811 03:15:54.635637 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:54.635717 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":346,"threads_started":1}
I20260811 03:15:55.135388 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:55.134643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:51141)
0811 03:15:55.135127 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:55.135209 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":253,"threads_started":1}
I20260811 03:15:55.616581 30449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:55.615847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:37247)
0811 03:15:55.616314 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:55.616391 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":240,"threads_started":1}
I20260811 03:15:55.637651 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:55.636756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:58991)
0811 03:15:55.637308 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:55.637427 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":309,"threads_started":1}
I20260811 03:15:56.137115 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:56.136348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:50291)
0811 03:15:56.136841 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:56.136944 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":258,"threads_started":1}
I20260811 03:15:56.618359 30453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:56.617547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:34633)
0811 03:15:56.618054 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:56.618145 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":278,"threads_started":1}
I20260811 03:15:56.639492 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:56.638685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:56855)
0811 03:15:56.639192 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:56.639272 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":226,"threads_started":1}
I20260811 03:15:57.139180 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:57.138195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:39943)
0811 03:15:57.138839 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:57.138928 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":337,"threads_started":1}
I20260811 03:15:57.620216 30456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:57.619336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:60989)
0811 03:15:57.619913 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:57.620004 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":280,"threads_started":1}
I20260811 03:15:57.641499 30457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:57.640611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:53111)
0811 03:15:57.641242 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:57.641328 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":364,"threads_started":1}
I20260811 03:15:58.141353 30458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:58.140363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:57399)
0811 03:15:58.141033 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:58.141124 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":337,"threads_started":1}
I20260811 03:15:58.621966 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:58.621252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:57705)
0811 03:15:58.621723 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:58.621806 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
I20260811 03:15:58.643468 30461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:58.642611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:41133)
0811 03:15:58.643204 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:58.643285 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":304,"threads_started":1}
I20260811 03:15:59.143149 30462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:59.142288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:42379)
0811 03:15:59.142871 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:59.142955 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":234,"threads_started":1}
I20260811 03:15:59.623781 30463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:59.622938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:45629)
0811 03:15:59.623511 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:59.623604 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":298,"threads_started":1}
I20260811 03:15:59.645136 30464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:15:59.644441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:35429)
0811 03:15:59.644881 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 03:15:59.644966 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":237,"threads_started":1}
I20260811 03:16:00.144932 30465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:00.144192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:43969)
0811 03:16:00.144670 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:00.144753 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":262,"threads_started":1}
I20260811 03:16:00.625608 30466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:00.624868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:33933)
0811 03:16:00.625344 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:00.625438 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":250,"threads_started":1}
I20260811 03:16:00.646934 30468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:00.646132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:34651)
0811 03:16:00.646662 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:00.646754 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":309,"threads_started":1}
I20260811 03:16:01.146879 30469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:01.145936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:44381)
0811 03:16:01.146562 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:01.146673 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":283,"threads_started":1}
I20260811 03:16:01.627444 30470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:01.626666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:37209)
0811 03:16:01.627144 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:01.627241 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":247,"threads_started":1}
I20260811 03:16:01.648524 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:01.647845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:34751)
0811 03:16:01.648276 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:01.648357 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":220,"threads_started":1}
I20260811 03:16:02.148731 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:02.147988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:53101)
0811 03:16:02.148457 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:02.148546 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":246,"threads_started":1}
I20260811 03:16:02.629232 30473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:02.628514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:60671)
0811 03:16:02.628987 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:02.629074 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":233,"threads_started":1}
I20260811 03:16:02.650581 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:02.649650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:46509)
0811 03:16:02.650176 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:02.650282 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":253,"threads_started":1}
I20260811 03:16:03.150712 30475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:03.149722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:35627)
0811 03:16:03.150262 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:03.150448 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":270,"threads_started":1}
I20260811 03:16:03.631346 30476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:03.630432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:51467)
0811 03:16:03.631054 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:03.631169 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":380,"threads_started":1}
I20260811 03:16:03.652391 30478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:03.651578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:55325)
0811 03:16:03.652045 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:03.652130 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":223,"threads_started":1}
I20260811 03:16:04.152552 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:04.151720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:37445)
0811 03:16:04.152222 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:04.152308 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":271,"threads_started":1}
I20260811 03:16:04.633035 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:04.632343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:35731)
0811 03:16:04.632770 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:04.632851 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":214,"threads_started":1}
I20260811 03:16:04.654385 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:04.653611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:55967)
0811 03:16:04.654045 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:04.654142 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":226,"threads_started":1}
I20260811 03:16:05.154822 30482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:05.153901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.2:57567)
0811 03:16:05.154475 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:05.154622 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":289,"threads_started":1}
I20260811 03:16:05.614759 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 29939
I20260811 03:16:05.634797 30483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:05.616131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45644 (local address 127.11.60.1:44885)
0811 03:16:05.616849 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:05.616866 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:05.616907 (+ 41us) server_negotiation.cc:407] Connection header received
0811 03:16:05.618736 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:05.618746 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:05.619275 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:05.619650 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:05.621286 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:05.622661 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:05.623972 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:05.624703 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:05.624918 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:05.627833 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:05.627841 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:05.627858 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:05.627896 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:05.630986 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:05.631729 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:05.631736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:05.631743 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:05.631918 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:05.632379 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:05.632385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:05.632390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:05.632944 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:05.633257 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:05.633775 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 03:16:05.634265 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":338,"threads_started":1}
I20260811 03:16:05.635131 30484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:05.634254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.3:56033)
0811 03:16:05.634762 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:05.634899 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":244,"threads_started":1}
I20260811 03:16:05.636317 30125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:05.725921 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 29939
I20260811 03:16:05.726012 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:05.724736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40559 (local address 127.11.60.1:33689)
0811 03:16:05.725409 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:05.725611 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.62:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":326,"threads_started":1}
I20260811 03:16:05.752151 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 30133
I20260811 03:16:05.771755 30487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:05.753172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56238 (local address 127.11.60.2:38241)
0811 03:16:05.753765 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:05.753780 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:05.753819 (+ 39us) server_negotiation.cc:407] Connection header received
0811 03:16:05.755514 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:05.755525 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:05.755990 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:05.756324 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:05.758000 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:05.759446 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:05.760891 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:05.761615 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:05.761765 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:05.764787 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:05.764795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:05.764812 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:05.764885 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:05.768191 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:05.769022 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:05.769030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:05.769037 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:05.769219 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:05.769655 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:05.769658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:05.769660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:05.770215 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:05.770534 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:05.770973 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 03:16:05.771442 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":301,"threads_started":1}
I20260811 03:16:05.773483 30271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:05.860625 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 30133
I20260811 03:16:05.882050 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 30280
I20260811 03:16:05.903739 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:05.883255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.11.60.3:43271)
0811 03:16:05.883965 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:05.883981 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:05.884027 (+ 46us) server_negotiation.cc:407] Connection header received
0811 03:16:05.885874 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:05.885886 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:05.886684 (+ 798us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:05.887098 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:05.888696 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:05.890043 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:05.891852 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:05.892905 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:05.893181 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:05.896317 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:05.896326 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:05.896344 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:05.896402 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:05.899749 (+ 3347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:05.900610 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:05.900622 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:05.900629 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:05.900831 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:05.901268 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:05.901274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:05.901278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:05.901916 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:05.902284 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:05.902877 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 03:16:05.903395 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":318,"threads_started":1}
I20260811 03:16:05.905293 30422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:05.992391 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 30280
2026-08-11T03:16:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64778 ms)
[----------] 30 tests from SecurityITest (307678 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:06 dist-test-slave-tm1g krb5kdc[30494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:06 dist-test-slave-tm1g krb5kdc[30494](info): set up 2 sockets
Aug 11 03:16:06 dist-test-slave-tm1g krb5kdc[30494](info): commencing operation
krb5kdc: starting...
W20260811 03:16:07.218924 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:07 dist-test-slave-tm1g krb5kdc[30494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:16:07.538228 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:37047
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:37047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:07.894147 30508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:07.894547 30508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:07.894666 30508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:07.902070 30508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:07.902191 30508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:07.902240 30508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:07.902290 30508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:07.902407 30508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:07.912863 30508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:37047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:37047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:07.914422 30508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:07.915715 30508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:07.928891 30513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:07.928996 30508 server_base.cc:1034] running on GCE node
W20260811 03:16:07.928856 30514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:07.928828 30516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:07.930215 30508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:07.931733 30508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:07.932984 30508 hybrid_clock.cc:648] HybridClock initialized: now 1786418167932941 us; error 33 us; skew 500 ppm
I20260811 03:16:07.933341 30508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:07 dist-test-slave-tm1g krb5kdc[30494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:07.936755 30508 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:16:07.938421 30508 webserver.cc:460] Webserver started at http://127.11.60.62:36333/ using document root <none> and password file <none>
I20260811 03:16:07.939108 30508 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:07.939215 30508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:07.939649 30508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:07.942902 30508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "b0d7245585a143d598375376625fbeff"
format_stamp: "Formatted at 2026-08-11 03:16:07 on dist-test-slave-tm1g"
server_key: "8866d9bedd01ff057ab2e87934afa5c3"
server_key_iv: "0c5a1011bc5e3694c646fc42c28528b1"
server_key_version: "encryptionkey@0"
I20260811 03:16:07.943629 30508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "b0d7245585a143d598375376625fbeff"
format_stamp: "Formatted at 2026-08-11 03:16:07 on dist-test-slave-tm1g"
server_key: "8866d9bedd01ff057ab2e87934afa5c3"
server_key_iv: "0c5a1011bc5e3694c646fc42c28528b1"
server_key_version: "encryptionkey@0"
I20260811 03:16:07.949066 30508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:16:07.952653 30523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:07.954082 30508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:16:07.954236 30508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "b0d7245585a143d598375376625fbeff"
format_stamp: "Formatted at 2026-08-11 03:16:07 on dist-test-slave-tm1g"
server_key: "8866d9bedd01ff057ab2e87934afa5c3"
server_key_iv: "0c5a1011bc5e3694c646fc42c28528b1"
server_key_version: "encryptionkey@0"
I20260811 03:16:07.954445 30508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:07.973572 30508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:07.978180 30508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:07.978631 30508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:07.998749 30508 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:37047
I20260811 03:16:07.998836 30583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:37047 every 8 connection(s)
I20260811 03:16:08.000238 30508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:08.004356 30584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:08.009656 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 30508
I20260811 03:16:08.009904 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:08.010794 11504 external_mini_cluster.cc:1428] Setting key a24cf394f72bd52f5098c2531e858fe9
I20260811 03:16:08.016512 30584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff: Bootstrap starting.
I20260811 03:16:08.020370 30584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:08.021654 30584 log.cc:824] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:08.025434 30584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff: No bootstrap required, opened a new log
Aug 11 03:16:08 dist-test-slave-tm1g krb5kdc[30494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:08.033855 30584 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7245585a143d598375376625fbeff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37047 } }
I20260811 03:16:08.034289 30584 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:08.034468 30584 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0d7245585a143d598375376625fbeff, State: Initialized, Role: FOLLOWER
I20260811 03:16:08.035192 30584 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7245585a143d598375376625fbeff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37047 } }
I20260811 03:16:08.035486 30584 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:08.035629 30584 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:08.035858 30584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:08.036629 30587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:08.013085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38860 (local address 127.11.60.62:37047)
0811 03:16:08.014010 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:08.014021 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:08.014076 (+ 55us) server_negotiation.cc:407] Connection header received
0811 03:16:08.015876 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:08.015899 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:08.016295 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:08.016674 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:08.018549 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:08.019885 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:08.021975 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:08.023161 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:08.023336 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:08.028269 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:08.028277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:08.028292 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:08.028337 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:08.031781 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:08.032595 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:08.032608 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:08.032625 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:08.032815 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:08.033245 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:08.033251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:08.033254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:08.033811 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:08.034153 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:08.034669 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 03:16:08.035141 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":421,"threads_started":1}
I20260811 03:16:08.038760 30584 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7245585a143d598375376625fbeff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37047 } }
I20260811 03:16:08.039273 30584 leader_election.cc:302] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0d7245585a143d598375376625fbeff; no voters:
I20260811 03:16:08.039918 30584 leader_election.cc:288] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:08.040046 30590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:08.040867 30590 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [term 1 LEADER]: Becoming Leader. State: Replica: b0d7245585a143d598375376625fbeff, State: Running, Role: LEADER
I20260811 03:16:08.041461 30590 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7245585a143d598375376625fbeff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37047 } }
I20260811 03:16:08.042125 30584 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:08.044315 30591 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0d7245585a143d598375376625fbeff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7245585a143d598375376625fbeff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37047 } } }
I20260811 03:16:08.044632 30591 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:08.045718 30592 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0d7245585a143d598375376625fbeff. Latest consensus state: current_term: 1 leader_uuid: "b0d7245585a143d598375376625fbeff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d7245585a143d598375376625fbeff" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 37047 } } }
I20260811 03:16:08.046144 30592 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:08.047232 30597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:08.052722 30597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:08.064452 30597 catalog_manager.cc:1347] Generated new cluster ID: 1072aad36adf4561b20f39ae8e9f94b5
I20260811 03:16:08.064656 30597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:08.082518 30597 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:08.083863 30597 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:08.091769 30597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff: Generated new TSK 0
I20260811 03:16:08.092906 30597 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:16:08.186827 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37047
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:08.543735 30615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:08.544040 30615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:08.544142 30615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:08.551738 30615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:08.551849 30615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:08.551970 30615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:16:08.561896 30615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:08.563184 30615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:08.564528 30615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:08.578363 30622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:08.578620 30624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:08.578831 30621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:08.579308 30615 server_base.cc:1034] running on GCE node
I20260811 03:16:08.579869 30615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:08.581143 30615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:08.582461 30615 hybrid_clock.cc:648] HybridClock initialized: now 1786418168582378 us; error 80 us; skew 500 ppm
I20260811 03:16:08.582803 30615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:08 dist-test-slave-tm1g krb5kdc[30494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418168, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:08.590974 30615 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:16:08.592700 30615 webserver.cc:460] Webserver started at http://127.11.60.1:46305/ using document root <none> and password file <none>
I20260811 03:16:08.593446 30615 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:08.593583 30615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:08.593938 30615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:08.597088 30615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "d43227613cdc4b45b67e396fabc8dd87"
format_stamp: "Formatted at 2026-08-11 03:16:08 on dist-test-slave-tm1g"
server_key: "399de6ddc40e9cf059336b4a4cd60a1a"
server_key_iv: "d2218100e30ccdd6714afefb34a65900"
server_key_version: "encryptionkey@0"
I20260811 03:16:08.597889 30615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "d43227613cdc4b45b67e396fabc8dd87"
format_stamp: "Formatted at 2026-08-11 03:16:08 on dist-test-slave-tm1g"
server_key: "399de6ddc40e9cf059336b4a4cd60a1a"
server_key_iv: "d2218100e30ccdd6714afefb34a65900"
server_key_version: "encryptionkey@0"
I20260811 03:16:08.603600 30615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 03:16:08.607331 30632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:08.608848 30615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:16:08.609045 30615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "d43227613cdc4b45b67e396fabc8dd87"
format_stamp: "Formatted at 2026-08-11 03:16:08 on dist-test-slave-tm1g"
server_key: "399de6ddc40e9cf059336b4a4cd60a1a"
server_key_iv: "d2218100e30ccdd6714afefb34a65900"
server_key_version: "encryptionkey@0"
I20260811 03:16:08.609269 30615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:08.631083 30615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:08.635519 30615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:08.635875 30615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:08.636968 30615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:08.638401 30615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:08.638511 30615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:08.638667 30615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:08.638742 30615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:08.676157 30615 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:34521
I20260811 03:16:08.676252 30760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:34521 every 8 connection(s)
I20260811 03:16:08.677510 30615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:16:08.680222 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 30615
I20260811 03:16:08.680464 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:16:08.681309 11504 external_mini_cluster.cc:1428] Setting key 13b7ccf7ee24b6da7319416066fc2030
Aug 11 03:16:08 dist-test-slave-tm1g krb5kdc[30494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418168, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:08.700124 30763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:08.680264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:52343 (local address 127.11.60.62:37047)
0811 03:16:08.680855 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:08.680862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:08.681939 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 03:16:08.684336 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:08.684343 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:08.684507 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:08.684629 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:08.686240 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:08.687468 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:08.689113 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:08.689847 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:08.690004 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:08.693707 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:08.693712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:08.693716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:08.693752 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:08.696373 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:08.697380 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:08.697387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:08.697393 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:08.697564 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:08.698094 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:08.698098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:08.698100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:08.698517 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:08.698649 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:08.699645 (+ 996us) server_negotiation.cc:299] Negotiation successful
0811 03:16:08.699836 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":268,"threads_started":1}
I20260811 03:16:08.701673 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:08.680857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37047 (local address 127.11.60.1:52343)
0811 03:16:08.681736 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:08.681795 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:08.684060 (+ 2265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:08.684889 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:08.684897 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:08.685458 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:08.686043 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:08.686061 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:08.687652 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:08.687656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:08.688953 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:08.688966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:08.690100 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:08.690105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:08.690418 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:08.691147 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:16:08.691177 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:08.693464 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:08.696560 (+ 3096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:08.696570 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:08.696592 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:08.697241 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:08.697694 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:08.697701 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:08.697706 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:08.697956 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:08.698652 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:08.698663 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:08.699152 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 03:16:08.699463 (+ 311us) client_negotiation.cc:240] Negotiation successful
0811 03:16:08.700085 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":315,"threads_started":1}
I20260811 03:16:08.703507 30761 heartbeater.cc:347] Connected to a master server at 127.11.60.62:37047
I20260811 03:16:08.703823 30761 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:08.704576 30761 heartbeater.cc:511] Master 127.11.60.62:37047 requested a full tablet report, sending...
I20260811 03:16:08.707005 30545 ts_manager.cc:194] Registered new tserver with Master: d43227613cdc4b45b67e396fabc8dd87 (127.11.60.1:34521)
I20260811 03:16:08.709301 30545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:52343
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:16:08.754490 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37047
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:09.103533 30767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:09.103824 30767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:09.103920 30767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:09.111452 30767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:09.111565 30767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:09.111680 30767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:16:09.121682 30767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:09.123057 30767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:09.124325 30767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:09.138501 30772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:09.138535 30776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:09.138446 30774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:09.139544 30767 server_base.cc:1034] running on GCE node
I20260811 03:16:09.140189 30767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:09.141098 30767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:09.142408 30767 hybrid_clock.cc:648] HybridClock initialized: now 1786418169142353 us; error 73 us; skew 500 ppm
I20260811 03:16:09.142781 30767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:09 dist-test-slave-tm1g krb5kdc[30494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418169, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:09.146139 30767 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:16:09.147768 30767 webserver.cc:460] Webserver started at http://127.11.60.2:45475/ using document root <none> and password file <none>
I20260811 03:16:09.148461 30767 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:09.148566 30767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:09.148828 30767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:09.151861 30767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "4e27bca2c7de4e3b8af0eada494423ff"
format_stamp: "Formatted at 2026-08-11 03:16:09 on dist-test-slave-tm1g"
server_key: "6aad21cc1074d6d762f9ad1fa5331227"
server_key_iv: "4f80be72733c036bad8015c876d5d827"
server_key_version: "encryptionkey@0"
I20260811 03:16:09.152607 30767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "4e27bca2c7de4e3b8af0eada494423ff"
format_stamp: "Formatted at 2026-08-11 03:16:09 on dist-test-slave-tm1g"
server_key: "6aad21cc1074d6d762f9ad1fa5331227"
server_key_iv: "4f80be72733c036bad8015c876d5d827"
server_key_version: "encryptionkey@0"
I20260811 03:16:09.158470 30767 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 03:16:09.162057 30783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:09.163515 30767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:16:09.163659 30767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "4e27bca2c7de4e3b8af0eada494423ff"
format_stamp: "Formatted at 2026-08-11 03:16:09 on dist-test-slave-tm1g"
server_key: "6aad21cc1074d6d762f9ad1fa5331227"
server_key_iv: "4f80be72733c036bad8015c876d5d827"
server_key_version: "encryptionkey@0"
I20260811 03:16:09.163849 30767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:09.192374 30767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:09.197044 30767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:09.197314 30767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:09.198374 30767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:09.199759 30767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:09.199836 30767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:09.199921 30767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:09.199965 30767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:09.235172 30767 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:34777
I20260811 03:16:09.235224 30909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:34777 every 8 connection(s)
I20260811 03:16:09.236584 30767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:16:09.246647 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 30767
I20260811 03:16:09.246855 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:16:09.247478 11504 external_mini_cluster.cc:1428] Setting key 40870be63a5efcfd48d387358f19380d
Aug 11 03:16:09 dist-test-slave-tm1g krb5kdc[30494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418169, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:09.258226 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.239443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:57173 (local address 127.11.60.62:37047)
0811 03:16:09.239932 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:09.239938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:09.241019 (+ 1081us) server_negotiation.cc:407] Connection header received
0811 03:16:09.242985 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:09.242990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:09.243070 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:09.243193 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:09.244666 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.245407 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.247120 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.247792 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.247955 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.251725 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:09.251735 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:09.251743 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:09.251790 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:09.255069 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:09.255789 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:09.255793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:09.255795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:09.255932 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:09.256374 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:09.256384 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:09.256388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:09.256661 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:09.256819 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:09.257540 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 03:16:09.257829 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":226,"threads_started":1}
I20260811 03:16:09.259373 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.240036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37047 (local address 127.11.60.2:57173)
0811 03:16:09.240834 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:09.240878 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:09.242700 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:09.243413 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:09.243421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:09.243952 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:09.244511 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:09.244527 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.245543 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.245547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:09.246967 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:09.246987 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.248018 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.248022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:09.248304 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.249038 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:16:09.249067 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:09.251482 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:09.255219 (+ 3737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:09.255226 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:09.255239 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:09.255646 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:09.256047 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:09.256052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:09.256055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:09.256250 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:09.256791 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:09.256799 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:09.257147 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 03:16:09.257421 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 03:16:09.258021 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":361,"threads_started":1}
I20260811 03:16:09.260926 30910 heartbeater.cc:347] Connected to a master server at 127.11.60.62:37047
I20260811 03:16:09.261246 30910 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:09.261989 30910 heartbeater.cc:511] Master 127.11.60.62:37047 requested a full tablet report, sending...
I20260811 03:16:09.263701 30545 ts_manager.cc:194] Registered new tserver with Master: 4e27bca2c7de4e3b8af0eada494423ff (127.11.60.2:34777)
I20260811 03:16:09.264930 30545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:57173
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:16:09.334177 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37047
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:09.689750 30916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:09.690017 30916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:09.690184 30916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:09.697496 30916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:09.697610 30916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:09.697795 30916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:16:09.707647 30916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:37047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:09.708977 30916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:09.710397 30916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 03:16:09.712641 30761 heartbeater.cc:503] Master 127.11.60.62:37047 was elected leader, sending a full tablet report...
W20260811 03:16:09.724205 30922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:09.724288 30925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:09.724255 30923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:09.725385 30916 server_base.cc:1034] running on GCE node
I20260811 03:16:09.726030 30916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:09.727082 30916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:09.728332 30916 hybrid_clock.cc:648] HybridClock initialized: now 1786418169728271 us; error 56 us; skew 500 ppm
I20260811 03:16:09.728677 30916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:09 dist-test-slave-tm1g krb5kdc[30494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418169, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:09.732165 30916 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:16:09.733788 30916 webserver.cc:460] Webserver started at http://127.11.60.3:39457/ using document root <none> and password file <none>
I20260811 03:16:09.734632 30916 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:09.734781 30916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:09.735113 30916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:09.738090 30916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "083f3f2fbf8e41a081ad995409f0568c"
format_stamp: "Formatted at 2026-08-11 03:16:09 on dist-test-slave-tm1g"
server_key: "567f11b9f2369e735df56e1560c6dc86"
server_key_iv: "ce487753c07bd21fa0067734cfd1efb0"
server_key_version: "encryptionkey@0"
I20260811 03:16:09.739001 30916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "083f3f2fbf8e41a081ad995409f0568c"
format_stamp: "Formatted at 2026-08-11 03:16:09 on dist-test-slave-tm1g"
server_key: "567f11b9f2369e735df56e1560c6dc86"
server_key_iv: "ce487753c07bd21fa0067734cfd1efb0"
server_key_version: "encryptionkey@0"
I20260811 03:16:09.746644 30916 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 03:16:09.750692 30933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:09.752238 30916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:16:09.752429 30916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "083f3f2fbf8e41a081ad995409f0568c"
format_stamp: "Formatted at 2026-08-11 03:16:09 on dist-test-slave-tm1g"
server_key: "567f11b9f2369e735df56e1560c6dc86"
server_key_iv: "ce487753c07bd21fa0067734cfd1efb0"
server_key_version: "encryptionkey@0"
I20260811 03:16:09.752660 30916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:09.775166 30916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:09.779543 30916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:09.779844 30916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:09.780881 30916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:09.782289 30916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:09.782449 30916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:09.782608 30916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:09.782686 30916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:09.816479 30916 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:33251
I20260811 03:16:09.816581 31070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:33251 every 8 connection(s)
I20260811 03:16:09.817903 30916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:16:09.826803 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 30916
I20260811 03:16:09.827031 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:16:09.827476 11504 external_mini_cluster.cc:1428] Setting key 7c553b93d81cb45977df443f4aecf6ac
Aug 11 03:16:09 dist-test-slave-tm1g krb5kdc[30494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418169, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:09.838073 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.820636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:39681 (local address 127.11.60.62:37047)
0811 03:16:09.821152 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:09.821157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:09.822126 (+ 969us) server_negotiation.cc:407] Connection header received
0811 03:16:09.824046 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:09.824051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:09.824139 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:09.824239 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:09.825788 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.826669 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.828432 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.828854 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.828990 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.832496 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:09.832501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:09.832504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:09.832537 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:09.835068 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:09.835770 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:09.835782 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:09.835785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:09.835907 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:09.836330 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:09.836335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:09.836338 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:09.836629 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:09.836798 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:09.837466 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 03:16:09.837685 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":267,"threads_started":1}
I20260811 03:16:09.839066 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.821162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:37047 (local address 127.11.60.3:39681)
0811 03:16:09.821952 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:09.821998 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:09.823750 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:09.824482 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:09.824490 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:09.825026 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:09.825624 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:09.825642 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.826827 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.826832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:09.828255 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:09.828268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.829088 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.829097 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:09.829361 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.830056 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:16:09.830083 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:09.832275 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:09.835205 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:09.835214 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:09.835229 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:09.835640 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:09.836018 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:09.836024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:09.836028 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:09.836213 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:09.836762 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:09.836770 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:09.837095 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 03:16:09.837361 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 03:16:09.837855 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":340,"threads_started":1}
I20260811 03:16:09.840402 31071 heartbeater.cc:347] Connected to a master server at 127.11.60.62:37047
I20260811 03:16:09.840709 31071 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:09.841496 31071 heartbeater.cc:511] Master 127.11.60.62:37047 requested a full tablet report, sending...
I20260811 03:16:09.843058 30545 ts_manager.cc:194] Registered new tserver with Master: 083f3f2fbf8e41a081ad995409f0568c (127.11.60.3:33251)
I20260811 03:16:09.843990 30545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:39681
I20260811 03:16:09.855289 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:16:09.875759 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.860872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.11.60.62:37047)
0811 03:16:09.861100 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:09.861104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:09.861690 (+ 586us) server_negotiation.cc:407] Connection header received
0811 03:16:09.861876 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:09.861880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:09.861949 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:09.862037 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:09.863765 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.864579 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.866294 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.866799 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.866939 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.869782 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:09.869787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:09.869791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:09.869843 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:09.872734 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:09.873535 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:09.873539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:09.873542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:09.873669 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:09.874132 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:09.874135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:09.874137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:09.874545 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:09.874750 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:09.875169 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 03:16:09.875353 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 03:16:09.880333 30545 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:16:09.888055 30545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:16:09.914323 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.901425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:33251 (local address 127.0.0.1:60950)
0811 03:16:09.902977 (+ 1552us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:09.903033 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:09.903279 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:09.904820 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:09.904829 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:09.904867 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:09.905393 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:09.905416 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.907956 (+ 2540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.907965 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:09.910896 (+ 2931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:09.910916 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.912973 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.912979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:09.913323 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.913384 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 03:16:09.913521 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 03:16:09.913722 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1183,"mutex_wait_us":1,"spinlock_wait_cycles":4864,"thread_start_us":985,"threads_started":1}
I20260811 03:16:09.915122 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.901807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60950 (local address 127.11.60.3:33251)
0811 03:16:09.904134 (+ 2327us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:09.904144 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:09.904182 (+ 38us) server_negotiation.cc:407] Connection header received
0811 03:16:09.904283 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:09.904291 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:09.904588 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:09.904788 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:09.905707 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.907756 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.911171 (+ 3415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.912724 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.912900 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.913168 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:09.914590 (+ 1422us) server_negotiation.cc:299] Negotiation successful
0811 03:16:09.914771 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2034,"thread_start_us":1862,"threads_started":1}
I20260811 03:16:09.915822 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.902333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51252 (local address 127.11.60.1:34521)
0811 03:16:09.904369 (+ 2036us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:09.904376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:09.904926 (+ 550us) server_negotiation.cc:407] Connection header received
0811 03:16:09.905278 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:09.905289 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:09.905526 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:09.905742 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:09.907619 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.909720 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.912970 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.914375 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.914503 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.914741 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:09.915314 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 03:16:09.915487 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1884,"thread_start_us":1698,"threads_started":1}
I20260811 03:16:09.916653 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.901676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34521 (local address 127.0.0.1:51252)
0811 03:16:09.904810 (+ 3134us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:09.904828 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:09.905073 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:09.906776 (+ 1703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:09.906783 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:09.906822 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:09.907381 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:09.907396 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.909971 (+ 2575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.909978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:09.912741 (+ 2763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:09.912754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.914896 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.914902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:09.915092 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.915129 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 03:16:09.916089 (+ 960us) client_negotiation.cc:240] Negotiation successful
0811 03:16:09.916249 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3005,"mutex_wait_us":405,"spinlock_wait_cycles":5120,"thread_start_us":2842,"threads_started":1}
I20260811 03:16:09.920250 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.904904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:34777 (local address 127.0.0.1:35032)
0811 03:16:09.911720 (+ 6816us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:09.911738 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:09.911879 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:09.912522 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:09.912526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:09.912552 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:09.913020 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:09.913034 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.915461 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.915466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:09.917360 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:09.917377 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.919212 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.919428 (+ 216us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:09.919637 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.919671 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 03:16:09.919760 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 03:16:09.919893 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6696,"thread_start_us":6510,"threads_started":1}
I20260811 03:16:09.919866 30994 tablet_service.cc:1467] Processing CreateTablet for tablet c95540b49ca648cd8cfecc4a7176198b (DEFAULT_TABLE table=test-table [id=323558360cd045daba4d15c3ca883b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:09.920939 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:09.905445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.11.60.2:34777)
0811 03:16:09.911221 (+ 5776us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:09.911231 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:09.911871 (+ 640us) server_negotiation.cc:407] Connection header received
0811 03:16:09.912059 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:09.912068 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:09.912306 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:09.912508 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:09.913379 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.915205 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.917560 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:09.919005 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:09.919162 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:09.919371 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:09.920486 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0811 03:16:09.920684 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5598,"thread_start_us":5496,"threads_started":1}
I20260811 03:16:09.920512 30685 tablet_service.cc:1467] Processing CreateTablet for tablet c95540b49ca648cd8cfecc4a7176198b (DEFAULT_TABLE table=test-table [id=323558360cd045daba4d15c3ca883b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:09.921697 30994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c95540b49ca648cd8cfecc4a7176198b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:09.922014 30685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c95540b49ca648cd8cfecc4a7176198b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:09.923712 30838 tablet_service.cc:1467] Processing CreateTablet for tablet c95540b49ca648cd8cfecc4a7176198b (DEFAULT_TABLE table=test-table [id=323558360cd045daba4d15c3ca883b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:09.924747 30838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c95540b49ca648cd8cfecc4a7176198b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:09.936539 31096 tablet_bootstrap.cc:492] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: Bootstrap starting.
I20260811 03:16:09.939066 31097 tablet_bootstrap.cc:492] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: Bootstrap starting.
I20260811 03:16:09.940451 31096 tablet_bootstrap.cc:654] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:09.940879 31098 tablet_bootstrap.cc:492] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: Bootstrap starting.
I20260811 03:16:09.942224 31096 log.cc:824] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:09.942811 31097 tablet_bootstrap.cc:654] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:09.944094 31097 log.cc:824] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:09.944401 31098 tablet_bootstrap.cc:654] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:09.945685 31098 log.cc:824] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:09.945771 31096 tablet_bootstrap.cc:492] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: No bootstrap required, opened a new log
I20260811 03:16:09.946120 31096 ts_tablet_manager.cc:1397] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 03:16:09.947289 31097 tablet_bootstrap.cc:492] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: No bootstrap required, opened a new log
I20260811 03:16:09.947571 31097 ts_tablet_manager.cc:1397] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 03:16:09.948266 31098 tablet_bootstrap.cc:492] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: No bootstrap required, opened a new log
I20260811 03:16:09.948591 31098 ts_tablet_manager.cc:1397] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 03:16:09.956445 31098 raft_consensus.cc:348] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 } } peers { permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 } } peers { permanent_uuid: "4e27bca2c7de4e3b8af0eada494423ff" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34777 } }
I20260811 03:16:09.956950 31098 raft_consensus.cc:374] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:09.957069 31098 raft_consensus.cc:729] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083f3f2fbf8e41a081ad995409f0568c, State: Initialized, Role: FOLLOWER
I20260811 03:16:09.957175 31096 raft_consensus.cc:348] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 } } peers { permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 } } peers { permanent_uuid: "4e27bca2c7de4e3b8af0eada494423ff" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34777 } }
I20260811 03:16:09.957686 31096 raft_consensus.cc:374] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:09.957824 31096 raft_consensus.cc:729] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e27bca2c7de4e3b8af0eada494423ff, State: Initialized, Role: FOLLOWER
I20260811 03:16:09.957818 31098 consensus_queue.cc:260] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 } } peers { permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 } } peers { permanent_uuid: "4e27bca2c7de4e3b8af0eada494423ff" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34777 } }
I20260811 03:16:09.958451 31096 consensus_queue.cc:260] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 } } peers { permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 } } peers { permanent_uuid: "4e27bca2c7de4e3b8af0eada494423ff" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34777 } }
I20260811 03:16:09.958607 31097 raft_consensus.cc:348] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 } } peers { permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 } } peers { permanent_uuid: "4e27bca2c7de4e3b8af0eada494423ff" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34777 } }
I20260811 03:16:09.959157 31097 raft_consensus.cc:374] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:09.959287 31097 raft_consensus.cc:729] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d43227613cdc4b45b67e396fabc8dd87, State: Initialized, Role: FOLLOWER
I20260811 03:16:09.959399 31071 heartbeater.cc:503] Master 127.11.60.62:37047 was elected leader, sending a full tablet report...
I20260811 03:16:09.960103 31098 ts_tablet_manager.cc:1428] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: Time spent starting tablet: real 0.011s user 0.001s sys 0.011s
I20260811 03:16:09.960112 31097 consensus_queue.cc:260] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 } } peers { permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 } } peers { permanent_uuid: "4e27bca2c7de4e3b8af0eada494423ff" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34777 } }
I20260811 03:16:09.961192 30910 heartbeater.cc:503] Master 127.11.60.62:37047 was elected leader, sending a full tablet report...
I20260811 03:16:09.962357 31097 ts_tablet_manager.cc:1428] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260811 03:16:09.963687 31096 ts_tablet_manager.cc:1428] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: Time spent starting tablet: real 0.017s user 0.011s sys 0.003s
W20260811 03:16:09.988857 30911 tablet.cc:2369] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:16:10.028029 31103 raft_consensus.cc:482] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:16:10.028546 31103 raft_consensus.cc:504] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 } } peers { permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 } } peers { permanent_uuid: "4e27bca2c7de4e3b8af0eada494423ff" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34777 } }
I20260811 03:16:10.030396 31103 leader_election.cc:288] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 083f3f2fbf8e41a081ad995409f0568c (127.11.60.3:33251), d43227613cdc4b45b67e396fabc8dd87 (127.11.60.1:34521)
I20260811 03:16:10.037238 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:10.030784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:42181 (local address 127.11.60.3:33251)
0811 03:16:10.031036 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:10.031047 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:10.031709 (+ 662us) server_negotiation.cc:407] Connection header received
0811 03:16:10.031957 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:10.031961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:10.032021 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:10.032114 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:10.032689 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.033469 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.035469 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.036254 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.036421 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:10.036549 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:10.036835 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 03:16:10.036965 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 03:16:10.037329 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:10.030698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:33251 (local address 127.11.60.2:42181)
0811 03:16:10.031582 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:10.031601 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:10.031798 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:10.032139 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:10.032144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:10.032170 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:10.032535 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:10.032548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.033607 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.033613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:10.035302 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:10.035315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.036487 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.036493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:10.036664 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:10.036688 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:16:10.036810 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 03:16:10.036961 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":826,"threads_started":1}
I20260811 03:16:10.037516 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:10.031047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:42849 (local address 127.11.60.1:34521)
0811 03:16:10.031302 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:10.031311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:10.031998 (+ 687us) server_negotiation.cc:407] Connection header received
0811 03:16:10.032201 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:10.032205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:10.032265 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:10.032367 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:10.032976 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.033713 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.035697 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.036490 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.036672 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:10.036790 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:10.037116 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 03:16:10.037248 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 03:16:10.037951 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:10.030704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34521 (local address 127.11.60.2:42849)
0811 03:16:10.031876 (+ 1172us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:10.031892 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:10.032046 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:10.032384 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:10.032396 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:10.032426 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:10.032800 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:10.032808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.033864 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.033870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:10.035544 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:10.035556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.036790 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.036797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:10.036957 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:10.036979 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 03:16:10.037088 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 03:16:10.037210 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"mutex_wait_us":33,"thread_start_us":505,"threads_started":1}
I20260811 03:16:10.038473 31017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95540b49ca648cd8cfecc4a7176198b" candidate_uuid: "4e27bca2c7de4e3b8af0eada494423ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083f3f2fbf8e41a081ad995409f0568c" is_pre_election: true
I20260811 03:16:10.039044 31017 raft_consensus.cc:2454] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e27bca2c7de4e3b8af0eada494423ff in term 0.
I20260811 03:16:10.038968 30710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95540b49ca648cd8cfecc4a7176198b" candidate_uuid: "4e27bca2c7de4e3b8af0eada494423ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d43227613cdc4b45b67e396fabc8dd87" is_pre_election: true
I20260811 03:16:10.039466 30710 raft_consensus.cc:2454] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e27bca2c7de4e3b8af0eada494423ff in term 0.
I20260811 03:16:10.039731 30784 leader_election.cc:302] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 083f3f2fbf8e41a081ad995409f0568c, 4e27bca2c7de4e3b8af0eada494423ff; no voters:
I20260811 03:16:10.040145 31103 raft_consensus.cc:2781] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:16:10.040310 31103 raft_consensus.cc:482] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:16:10.040437 31103 raft_consensus.cc:3037] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:10.042935 31103 raft_consensus.cc:504] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 } } peers { permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 } } peers { permanent_uuid: "4e27bca2c7de4e3b8af0eada494423ff" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34777 } }
I20260811 03:16:10.043638 31103 leader_election.cc:288] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [CANDIDATE]: Term 1 election: Requested vote from peers 083f3f2fbf8e41a081ad995409f0568c (127.11.60.3:33251), d43227613cdc4b45b67e396fabc8dd87 (127.11.60.1:34521)
I20260811 03:16:10.044118 31017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95540b49ca648cd8cfecc4a7176198b" candidate_uuid: "4e27bca2c7de4e3b8af0eada494423ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083f3f2fbf8e41a081ad995409f0568c"
I20260811 03:16:10.044185 30710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95540b49ca648cd8cfecc4a7176198b" candidate_uuid: "4e27bca2c7de4e3b8af0eada494423ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d43227613cdc4b45b67e396fabc8dd87"
I20260811 03:16:10.044348 31017 raft_consensus.cc:3037] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:10.044373 30710 raft_consensus.cc:3037] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:10.047780 30710 raft_consensus.cc:2454] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e27bca2c7de4e3b8af0eada494423ff in term 1.
I20260811 03:16:10.047780 31017 raft_consensus.cc:2454] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e27bca2c7de4e3b8af0eada494423ff in term 1.
I20260811 03:16:10.048259 30784 leader_election.cc:302] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 083f3f2fbf8e41a081ad995409f0568c, 4e27bca2c7de4e3b8af0eada494423ff, d43227613cdc4b45b67e396fabc8dd87; no voters:
I20260811 03:16:10.048679 31103 raft_consensus.cc:2781] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:10.049186 31103 raft_consensus.cc:686] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 1 LEADER]: Becoming Leader. State: Replica: 4e27bca2c7de4e3b8af0eada494423ff, State: Running, Role: LEADER
I20260811 03:16:10.049712 31103 consensus_queue.cc:237] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 } } peers { permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 } } peers { permanent_uuid: "4e27bca2c7de4e3b8af0eada494423ff" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34777 } }
I20260811 03:16:10.054896 30544 catalog_manager.cc:5495] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e27bca2c7de4e3b8af0eada494423ff (127.11.60.2). New cstate: current_term: 1 leader_uuid: "4e27bca2c7de4e3b8af0eada494423ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e27bca2c7de4e3b8af0eada494423ff" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 34777 } health_report { overall_health: HEALTHY } } }
W20260811 03:16:10.069614 31072 tablet.cc:2369] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:16:10.108268 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:10.102232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35044 (local address 127.11.60.2:34777)
0811 03:16:10.102498 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:10.102504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:10.102620 (+ 116us) server_negotiation.cc:407] Connection header received
0811 03:16:10.102897 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:10.102901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:10.102980 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:10.103138 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:16:10.103559 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.104281 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.105746 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.106169 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.106305 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:10.107044 (+ 739us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:16:10.107613 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:16:10.107780 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:10.107890 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 03:16:10.107982 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 03:16:10.117447 31017 raft_consensus.cc:1261] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c [term 1 FOLLOWER]: Refusing update from remote peer 4e27bca2c7de4e3b8af0eada494423ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:10.117447 30710 raft_consensus.cc:1261] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87 [term 1 FOLLOWER]: Refusing update from remote peer 4e27bca2c7de4e3b8af0eada494423ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:10.118729 31108 consensus_queue.cc:1035] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "083f3f2fbf8e41a081ad995409f0568c" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:10.119124 31103 consensus_queue.cc:1035] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d43227613cdc4b45b67e396fabc8dd87" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:10.130257 31112 mvcc.cc:205] Tried to move back new op lower bound from 7317168824789004288 to 7317168824527806464. Current Snapshot: MvccSnapshot[applied={T|T < 7317168824789004288}]
I20260811 03:16:10.147804 30545 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:10.148139 30545 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:10.152019 30545 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0d7245585a143d598375376625fbeff: Sending DeleteTablet for 3 replicas of tablet c95540b49ca648cd8cfecc4a7176198b
I20260811 03:16:10.153367 30994 tablet_service.cc:1514] Processing DeleteTablet for tablet c95540b49ca648cd8cfecc4a7176198b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:10 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:60950
I20260811 03:16:10.153625 30685 tablet_service.cc:1514] Processing DeleteTablet for tablet c95540b49ca648cd8cfecc4a7176198b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:10 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:51252
I20260811 03:16:10.153663 30838 tablet_service.cc:1514] Processing DeleteTablet for tablet c95540b49ca648cd8cfecc4a7176198b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:10 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:35032
I20260811 03:16:10.156220 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 30615
I20260811 03:16:10.166944 31125 tablet_replica.cc:330] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: stopping tablet replica
I20260811 03:16:10.167593 31125 raft_consensus.cc:2229] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:16:10.168566 31125 raft_consensus.cc:2258] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:10.169559 31124 tablet_replica.cc:330] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: stopping tablet replica
I20260811 03:16:10.170058 31124 raft_consensus.cc:2229] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:10.170642 31124 raft_consensus.cc:2258] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:10.172005 31125 ts_tablet_manager.cc:1905] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:10.172930 31124 ts_tablet_manager.cc:1905] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:10.175501 31123 tablet_replica.cc:330] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: stopping tablet replica
I20260811 03:16:10.175920 31123 raft_consensus.cc:2229] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:10.176438 31123 raft_consensus.cc:2258] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:10.179819 31125 ts_tablet_manager.cc:1918] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:10.180019 31125 log.cc:1196] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/c95540b49ca648cd8cfecc4a7176198b
I20260811 03:16:10.180644 31124 ts_tablet_manager.cc:1918] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:10.180680 31125 ts_tablet_manager.cc:1939] T c95540b49ca648cd8cfecc4a7176198b P 4e27bca2c7de4e3b8af0eada494423ff: Deleting consensus metadata
I20260811 03:16:10.180938 31124 log.cc:1196] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/c95540b49ca648cd8cfecc4a7176198b
I20260811 03:16:10.181553 31124 ts_tablet_manager.cc:1939] T c95540b49ca648cd8cfecc4a7176198b P d43227613cdc4b45b67e396fabc8dd87: Deleting consensus metadata
I20260811 03:16:10.182253 31123 ts_tablet_manager.cc:1905] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:10.182648 30526 catalog_manager.cc:4841] TS 4e27bca2c7de4e3b8af0eada494423ff (127.11.60.2:34777): tablet c95540b49ca648cd8cfecc4a7176198b (table test-table [id=323558360cd045daba4d15c3ca883b2c]) successfully deleted
I20260811 03:16:10.183615 30525 catalog_manager.cc:4841] TS d43227613cdc4b45b67e396fabc8dd87 (127.11.60.1:34521): tablet c95540b49ca648cd8cfecc4a7176198b (table test-table [id=323558360cd045daba4d15c3ca883b2c]) successfully deleted
I20260811 03:16:10.187975 31123 ts_tablet_manager.cc:1918] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:10.188156 31123 log.cc:1196] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/c95540b49ca648cd8cfecc4a7176198b
Aug 11 03:16:10 dist-test-slave-tm1g krb5kdc[30494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:16:10.188714 31123 ts_tablet_manager.cc:1939] T c95540b49ca648cd8cfecc4a7176198b P 083f3f2fbf8e41a081ad995409f0568c: Deleting consensus metadata
I20260811 03:16:10.190532 30524 catalog_manager.cc:4841] TS 083f3f2fbf8e41a081ad995409f0568c (127.11.60.3:33251): tablet c95540b49ca648cd8cfecc4a7176198b (table test-table [id=323558360cd045daba4d15c3ca883b2c]) successfully deleted
I20260811 03:16:10.197486 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:10.157624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51266 (local address 127.11.60.1:34521)
0811 03:16:10.157931 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:10.157941 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:10.172508 (+ 14567us) server_negotiation.cc:407] Connection header received
0811 03:16:10.173033 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:10.173039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:10.173138 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:10.173213 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:10.176846 (+ 3633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.178068 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.183178 (+ 5110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.183837 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.183951 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:10.190000 (+ 6049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:10.190017 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:10.190037 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:10.190082 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:10.193967 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:10.194930 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:10.194939 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:10.194943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:10.195120 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:10.195755 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:10.195761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:10.195764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:10.196276 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:10.196615 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:10.196835 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 03:16:10.197091 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 03:16:10.198261 30754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:10.291649 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 30615
I20260811 03:16:10.319999 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 30767
Aug 11 03:16:10 dist-test-slave-tm1g krb5kdc[30494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:16:10.339871 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:10.321080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.11.60.2:34777)
0811 03:16:10.321276 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:10.321285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:10.321305 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:16:10.321370 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:10.321374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:10.321447 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:10.321540 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:10.323315 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.324286 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.326250 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.326839 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.326959 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:10.332673 (+ 5714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:10.332688 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:10.332706 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:10.332749 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:10.336641 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:10.337529 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:10.337540 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:10.337545 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:10.337727 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:10.338116 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:10.338121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:10.338125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:10.338612 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:10.338995 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:10.339214 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 03:16:10.339458 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 03:16:10.340790 30905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:10.457265 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 30767
I20260811 03:16:10.479979 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 30916
Aug 11 03:16:10 dist-test-slave-tm1g krb5kdc[30494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:16:10.496573 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:10.480834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.11.60.3:33251)
0811 03:16:10.481043 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:10.481049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:10.481157 (+ 108us) server_negotiation.cc:407] Connection header received
0811 03:16:10.481376 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:10.481383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:10.481515 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:10.481636 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:10.483082 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.483831 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.485138 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:10.485580 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:10.485721 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:10.490297 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:10.490359 (+ 62us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:10.490380 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:10.490467 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:10.493410 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:10.494180 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:10.494192 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:10.494200 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:10.494480 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:10.494885 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:10.494891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:10.494894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:10.495280 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:10.495612 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:10.495963 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 03:16:10.496201 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 03:16:10.497331 31065 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:10.590005 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 30916
I20260811 03:16:10.610410 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 30508
I20260811 03:16:10.611717 30579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:10.725214 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 30508
2026-08-11T03:16:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4738 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T03:16:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:10Z Disabled control of system clock
I20260811 03:16:10.822052 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:43109
--webserver_interface=127.11.60.62
--webserver_port=0
--builtin_ntp_servers=127.11.60.20:33757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:43109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:16:11.182580 31135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:11.182889 31135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:11.183089 31135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:11.191867 31135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:11.192008 31135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:11.192119 31135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:11.192209 31135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:11.192270 31135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:11.202291 31135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:43109
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:43109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:11.203708 31135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:11.205636 31135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:11.218035 31144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:11.217993 31142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:11.218082 31141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:11.219026 31135 server_base.cc:1034] running on GCE node
I20260811 03:16:11.219872 31135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:11.221398 31135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:11.222666 31135 hybrid_clock.cc:648] HybridClock initialized: now 1786418171222597 us; error 58 us; skew 500 ppm
I20260811 03:16:11.223060 31135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:11.225457 31135 webserver.cc:460] Webserver started at http://127.11.60.62:46023/ using document root <none> and password file <none>
I20260811 03:16:11.226197 31135 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:11.226354 31135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:11.226704 31135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:11.229722 31135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "9e410a7287eb45feb8449d940457d95c"
format_stamp: "Formatted at 2026-08-11 03:16:11 on dist-test-slave-tm1g"
server_key: "61dee3f8a6cb8ae15775ba81b6a6fb0f"
server_key_iv: "0e113cc0a31e64010a94a4c0aa09a2d2"
server_key_version: "encryptionkey@0"
I20260811 03:16:11.230623 31135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "9e410a7287eb45feb8449d940457d95c"
format_stamp: "Formatted at 2026-08-11 03:16:11 on dist-test-slave-tm1g"
server_key: "61dee3f8a6cb8ae15775ba81b6a6fb0f"
server_key_iv: "0e113cc0a31e64010a94a4c0aa09a2d2"
server_key_version: "encryptionkey@0"
I20260811 03:16:11.235916 31135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 03:16:11.239390 31157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:11.240669 31135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:16:11.240875 31135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "9e410a7287eb45feb8449d940457d95c"
format_stamp: "Formatted at 2026-08-11 03:16:11 on dist-test-slave-tm1g"
server_key: "61dee3f8a6cb8ae15775ba81b6a6fb0f"
server_key_iv: "0e113cc0a31e64010a94a4c0aa09a2d2"
server_key_version: "encryptionkey@0"
I20260811 03:16:11.241096 31135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:11.259800 31135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:11.260634 31135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:11.260926 31135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:11.284428 31135 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:43109
I20260811 03:16:11.284473 31218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:43109 every 8 connection(s)
I20260811 03:16:11.285790 31135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:11.289516 31219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:11.294574 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 31135
I20260811 03:16:11.294762 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:11.295346 11504 external_mini_cluster.cc:1428] Setting key 4bf4c9d28ce1a0cb7d5f90ab9c8cd125
I20260811 03:16:11.301971 31219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c: Bootstrap starting.
I20260811 03:16:11.305290 31219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:11.306628 31219 log.cc:824] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:11.310227 31219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c: No bootstrap required, opened a new log
I20260811 03:16:11.310437 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:11.297334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47542 (local address 127.11.60.62:43109)
0811 03:16:11.298438 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:11.298451 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:11.298493 (+ 42us) server_negotiation.cc:407] Connection header received
0811 03:16:11.300833 (+ 2340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:11.300863 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:11.301484 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:11.301902 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:11.302651 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:11.303973 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:11.305355 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:11.306123 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:11.306346 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:11.307689 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:11.307696 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:11.307710 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:11.308372 (+ 662us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:11.308482 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:11.308510 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:11.308632 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:11.308851 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 03:16:11.309174 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":372,"threads_started":1}
I20260811 03:16:11.318174 31219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e410a7287eb45feb8449d940457d95c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 43109 } }
I20260811 03:16:11.318678 31219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:11.318786 31219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e410a7287eb45feb8449d940457d95c, State: Initialized, Role: FOLLOWER
I20260811 03:16:11.319404 31219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e410a7287eb45feb8449d940457d95c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 43109 } }
I20260811 03:16:11.319643 31219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:11.319778 31219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:11.319954 31219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:11.322552 31219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e410a7287eb45feb8449d940457d95c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 43109 } }
I20260811 03:16:11.323058 31219 leader_election.cc:302] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e410a7287eb45feb8449d940457d95c; no voters:
I20260811 03:16:11.323637 31219 leader_election.cc:288] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:11.323765 31228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:11.324551 31228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [term 1 LEADER]: Becoming Leader. State: Replica: 9e410a7287eb45feb8449d940457d95c, State: Running, Role: LEADER
I20260811 03:16:11.325137 31228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e410a7287eb45feb8449d940457d95c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 43109 } }
I20260811 03:16:11.325847 31219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:11.327849 31230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e410a7287eb45feb8449d940457d95c. Latest consensus state: current_term: 1 leader_uuid: "9e410a7287eb45feb8449d940457d95c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e410a7287eb45feb8449d940457d95c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 43109 } } }
I20260811 03:16:11.328219 31230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:11.329523 31238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:11.329408 31229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e410a7287eb45feb8449d940457d95c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e410a7287eb45feb8449d940457d95c" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 43109 } } }
I20260811 03:16:11.329764 31229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:11.335755 31238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:11.346453 31238 catalog_manager.cc:1347] Generated new cluster ID: 270a63f547a94d21b96aff7baffda333
I20260811 03:16:11.346594 31238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:11.362912 31238 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:11.364450 31238 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:11.374558 31238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c: Generated new TSK 0
I20260811 03:16:11.375512 31238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:11.381234 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:43109
--builtin_ntp_servers=127.11.60.20:33757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:16:11.732338 31250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:11.732651 31250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:11.732810 31250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:11.740638 31250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:11.740790 31250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:11.740973 31250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:16:11.753299 31250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:43109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:11.754719 31250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:11.756505 31250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:11.770532 31255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:11.770430 31261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:11.770488 31256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:11.771337 31250 server_base.cc:1034] running on GCE node
I20260811 03:16:11.771958 31250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:11.773010 31250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:11.774245 31250 hybrid_clock.cc:648] HybridClock initialized: now 1786418171774199 us; error 55 us; skew 500 ppm
I20260811 03:16:11.774595 31250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:11.776882 31250 webserver.cc:460] Webserver started at http://127.11.60.1:40039/ using document root <none> and password file <none>
I20260811 03:16:11.777599 31250 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:11.777733 31250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:11.778062 31250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:11.781294 31250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "485c97cf3563432d8575a1c332a63478"
format_stamp: "Formatted at 2026-08-11 03:16:11 on dist-test-slave-tm1g"
server_key: "85eb26506aa356a46ac4f24840a5c92d"
server_key_iv: "8782dd8ffb526d81591884ceab9758bc"
server_key_version: "encryptionkey@0"
I20260811 03:16:11.782160 31250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "485c97cf3563432d8575a1c332a63478"
format_stamp: "Formatted at 2026-08-11 03:16:11 on dist-test-slave-tm1g"
server_key: "85eb26506aa356a46ac4f24840a5c92d"
server_key_iv: "8782dd8ffb526d81591884ceab9758bc"
server_key_version: "encryptionkey@0"
I20260811 03:16:11.787683 31250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 03:16:11.791563 31267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:11.792930 31250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 03:16:11.793128 31250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "485c97cf3563432d8575a1c332a63478"
format_stamp: "Formatted at 2026-08-11 03:16:11 on dist-test-slave-tm1g"
server_key: "85eb26506aa356a46ac4f24840a5c92d"
server_key_iv: "8782dd8ffb526d81591884ceab9758bc"
server_key_version: "encryptionkey@0"
I20260811 03:16:11.793354 31250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:11.812289 31250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:11.813515 31250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:11.813824 31250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:11.814992 31250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:11.816535 31250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:11.816680 31250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:11.816823 31250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:11.816918 31250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:11.852825 31250 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:34421
I20260811 03:16:11.852921 31396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:34421 every 8 connection(s)
I20260811 03:16:11.854219 31250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:16:11.854594 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 31250
I20260811 03:16:11.854807 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:16:11.855449 11504 external_mini_cluster.cc:1428] Setting key afc10c7a40897c8e40eed8626a8fe307
I20260811 03:16:11.858866 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:43109
--builtin_ntp_servers=127.11.60.20:33757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:16:11.869728 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:11.857141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:56543 (local address 127.11.60.62:43109)
0811 03:16:11.857637 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:11.857643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:11.862074 (+ 4431us) server_negotiation.cc:407] Connection header received
0811 03:16:11.862161 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:11.862170 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:11.862271 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:11.862409 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:11.863525 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:11.864688 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:11.866386 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:11.867101 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:11.867249 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:11.868726 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:11.868734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:11.868742 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:11.868769 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:11.868826 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:11.868845 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:11.868942 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:11.869161 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 03:16:11.869281 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":255,"threads_started":1}
I20260811 03:16:11.870710 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:11.857805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:43109 (local address 127.11.60.1:56543)
0811 03:16:11.858979 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:11.859030 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:11.861704 (+ 2674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:11.862624 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:11.862633 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:11.862719 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:11.863338 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:11.863357 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:11.864870 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:11.864874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:11.866198 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:11.866211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:11.867325 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:11.867330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:11.867538 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:11.868359 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:16:11.868362 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:11.868410 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:16:11.868523 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:11.868948 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:11.868956 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:11.868962 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 03:16:11.869118 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 03:16:11.869426 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":330,"threads_started":1}
I20260811 03:16:11.872575 31397 heartbeater.cc:347] Connected to a master server at 127.11.60.62:43109
I20260811 03:16:11.872931 31397 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:11.873822 31397 heartbeater.cc:511] Master 127.11.60.62:43109 requested a full tablet report, sending...
I20260811 03:16:11.876175 31176 ts_manager.cc:194] Registered new tserver with Master: 485c97cf3563432d8575a1c332a63478 (127.11.60.1:34421)
I20260811 03:16:11.877766 31176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.1:56543
W20260811 03:16:12.205461 31401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:12.205747 31401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:12.205897 31401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:12.213179 31401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:12.213277 31401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:12.213469 31401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:16:12.222648 31401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:43109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:12.224031 31401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:12.225905 31401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:12.239284 31410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:12.239327 31412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:12.239549 31409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:12.240262 31401 server_base.cc:1034] running on GCE node
I20260811 03:16:12.240921 31401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:12.242002 31401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:12.243204 31401 hybrid_clock.cc:648] HybridClock initialized: now 1786418172243159 us; error 32 us; skew 500 ppm
I20260811 03:16:12.243492 31401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:12.246090 31401 webserver.cc:460] Webserver started at http://127.11.60.2:43409/ using document root <none> and password file <none>
I20260811 03:16:12.246898 31401 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:12.247040 31401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:12.247364 31401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:12.250284 31401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "299f6970d13a452fa98652d19bbc0ad6"
format_stamp: "Formatted at 2026-08-11 03:16:12 on dist-test-slave-tm1g"
server_key: "d934dccdd458279636d9f246cb94f059"
server_key_iv: "8e7a5bacc1374022da64e05c1404948e"
server_key_version: "encryptionkey@0"
I20260811 03:16:12.251214 31401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "299f6970d13a452fa98652d19bbc0ad6"
format_stamp: "Formatted at 2026-08-11 03:16:12 on dist-test-slave-tm1g"
server_key: "d934dccdd458279636d9f246cb94f059"
server_key_iv: "8e7a5bacc1374022da64e05c1404948e"
server_key_version: "encryptionkey@0"
I20260811 03:16:12.256502 31401 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:16:12.259997 31419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:12.261226 31401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:16:12.261402 31401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "299f6970d13a452fa98652d19bbc0ad6"
format_stamp: "Formatted at 2026-08-11 03:16:12 on dist-test-slave-tm1g"
server_key: "d934dccdd458279636d9f246cb94f059"
server_key_iv: "8e7a5bacc1374022da64e05c1404948e"
server_key_version: "encryptionkey@0"
I20260811 03:16:12.261648 31401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:12.302105 31401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:12.303156 31401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:12.303548 31401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:12.304903 31401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:12.306905 31401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:12.307000 31401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:12.307091 31401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:12.307145 31401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:12.342383 31401 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:37377
I20260811 03:16:12.342459 31542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:37377 every 8 connection(s)
I20260811 03:16:12.343776 31401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:16:12.353083 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 31401
I20260811 03:16:12.353302 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:16:12.353964 11504 external_mini_cluster.cc:1428] Setting key f31ef6e7fe720dbc1cf3d86ce1beda73
I20260811 03:16:12.357383 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:43109
--builtin_ntp_servers=127.11.60.20:33757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:16:12.358770 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.346654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:35663 (local address 127.11.60.62:43109)
0811 03:16:12.346887 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:12.346893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:12.348398 (+ 1505us) server_negotiation.cc:407] Connection header received
0811 03:16:12.350427 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:12.350432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:12.350516 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:12.350619 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:12.351726 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.352656 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.355217 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.355949 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.356102 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.357817 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:12.357823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:12.357829 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:12.357858 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:12.357893 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:12.357907 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:12.358016 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:12.358275 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 03:16:12.358417 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":87}
I20260811 03:16:12.359932 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.347208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:43109 (local address 127.11.60.2:35663)
0811 03:16:12.348197 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:12.348244 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:12.350063 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:12.350824 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:12.350833 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:12.350950 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:12.351555 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.351572 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.352820 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.352828 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.355021 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.355038 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.356290 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.356298 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.356535 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.357356 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:16:12.357359 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:12.357403 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:16:12.357519 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:12.358019 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:12.358033 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:12.358045 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 03:16:12.358259 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 03:16:12.358738 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":315,"threads_started":1}
I20260811 03:16:12.361532 31545 heartbeater.cc:347] Connected to a master server at 127.11.60.62:43109
I20260811 03:16:12.361886 31545 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:12.362917 31545 heartbeater.cc:511] Master 127.11.60.62:43109 requested a full tablet report, sending...
I20260811 03:16:12.364701 31176 ts_manager.cc:194] Registered new tserver with Master: 299f6970d13a452fa98652d19bbc0ad6 (127.11.60.2:37377)
I20260811 03:16:12.366185 31176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.2:35663
W20260811 03:16:12.705209 31549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:12.705536 31549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:12.705626 31549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:12.712989 31549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:12.713107 31549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:12.713218 31549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:16:12.722740 31549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:43109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:12.724242 31549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:12.726363 31549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:12.740739 31554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:12.740940 31556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:12.741140 31558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:12.742095 31549 server_base.cc:1034] running on GCE node
I20260811 03:16:12.742807 31549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:12.743860 31549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:12.745106 31549 hybrid_clock.cc:648] HybridClock initialized: now 1786418172745060 us; error 75 us; skew 500 ppm
I20260811 03:16:12.745522 31549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:12.748121 31549 webserver.cc:460] Webserver started at http://127.11.60.3:41117/ using document root <none> and password file <none>
I20260811 03:16:12.748898 31549 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:12.749044 31549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:12.749392 31549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:12.752458 31549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "af4e9bfeb04942b587c5186641ad87ad"
format_stamp: "Formatted at 2026-08-11 03:16:12 on dist-test-slave-tm1g"
server_key: "78039ad80461872ad18775f21bee5305"
server_key_iv: "19771aa6ee4ef7e6fedbac69c58c77ed"
server_key_version: "encryptionkey@0"
I20260811 03:16:12.753361 31549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "af4e9bfeb04942b587c5186641ad87ad"
format_stamp: "Formatted at 2026-08-11 03:16:12 on dist-test-slave-tm1g"
server_key: "78039ad80461872ad18775f21bee5305"
server_key_iv: "19771aa6ee4ef7e6fedbac69c58c77ed"
server_key_version: "encryptionkey@0"
I20260811 03:16:12.758817 31549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 03:16:12.762547 31566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:12.763902 31549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:16:12.764099 31549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "af4e9bfeb04942b587c5186641ad87ad"
format_stamp: "Formatted at 2026-08-11 03:16:12 on dist-test-slave-tm1g"
server_key: "78039ad80461872ad18775f21bee5305"
server_key_iv: "19771aa6ee4ef7e6fedbac69c58c77ed"
server_key_version: "encryptionkey@0"
I20260811 03:16:12.764335 31549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:12.786201 31549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:12.787149 31549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:12.787456 31549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:12.788519 31549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:12.790050 31549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:12.790175 31549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:12.790333 31549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:12.790426 31549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 03:16:12.826426 31549 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:43321
I20260811 03:16:12.826524 31695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:43321 every 8 connection(s)
I20260811 03:16:12.827834 31549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:16:12.831451 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 31549
I20260811 03:16:12.831630 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:16:12.832250 11504 external_mini_cluster.cc:1428] Setting key 5229b0f22e4bad00fbad5fd831c4792f
I20260811 03:16:12.841482 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.830804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:39643 (local address 127.11.60.62:43109)
0811 03:16:12.831014 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:12.831019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:12.832617 (+ 1598us) server_negotiation.cc:407] Connection header received
0811 03:16:12.835258 (+ 2641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:12.835263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:12.835352 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:12.835449 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:12.836695 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.837440 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.839055 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.839475 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.839575 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.840843 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:12.840847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:12.840850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:12.840867 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:12.840896 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:12.840907 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:12.840973 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:12.841176 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 03:16:12.841248 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 03:16:12.842509 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.831254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:43109 (local address 127.11.60.3:39643)
0811 03:16:12.832426 (+ 1172us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:12.832481 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:12.834874 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:12.835904 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:12.835911 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:12.835992 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:12.836540 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.836555 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.837583 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.837586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.838923 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.838936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.839660 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.839664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.839856 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.840573 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:16:12.840575 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:12.840613 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:16:12.840689 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:12.840993 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:12.841001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:12.841007 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 03:16:12.841152 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 03:16:12.841433 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":402,"threads_started":1}
I20260811 03:16:12.843909 31696 heartbeater.cc:347] Connected to a master server at 127.11.60.62:43109
I20260811 03:16:12.844228 31696 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:12.845045 31696 heartbeater.cc:511] Master 127.11.60.62:43109 requested a full tablet report, sending...
I20260811 03:16:12.846664 31176 ts_manager.cc:194] Registered new tserver with Master: af4e9bfeb04942b587c5186641ad87ad (127.11.60.3:43321)
I20260811 03:16:12.847642 31176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.3:39643
I20260811 03:16:12.848295 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:16:12.859490 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.852629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.11.60.62:43109)
0811 03:16:12.852796 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:12.852816 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:12.853285 (+ 469us) server_negotiation.cc:407] Connection header received
0811 03:16:12.853479 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:12.853485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:12.853566 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:12.853666 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:12.854197 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.855082 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.857218 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.857619 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.857714 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.858927 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:12.858932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:12.858935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:12.858951 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:12.858982 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:12.859003 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:12.859068 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:12.859196 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 03:16:12.859275 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 03:16:12.863165 31176 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:16:12.870509 31176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:16:12.880863 31397 heartbeater.cc:503] Master 127.11.60.62:43109 was elected leader, sending a full tablet report...
I20260811 03:16:12.903362 31709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.882963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34421 (local address 127.0.0.1:57466)
0811 03:16:12.884071 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:12.884122 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:12.884284 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:12.895772 (+ 11488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:12.895776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:12.895801 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:12.896322 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.896335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.898355 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.898361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.900532 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.900550 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.902287 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.902293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.902538 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.902595 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 03:16:12.902749 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 03:16:12.902948 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":146,"thread_start_us":736,"threads_started":1}
I20260811 03:16:12.903347 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.883174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57466 (local address 127.11.60.1:34421)
0811 03:16:12.895152 (+ 11978us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:12.895162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:12.895195 (+ 33us) server_negotiation.cc:407] Connection header received
0811 03:16:12.895304 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:12.895311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:12.895553 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:12.895882 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:12.896520 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.898187 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.900803 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.902084 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.902227 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.902559 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:12.902801 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 03:16:12.902963 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11802,"thread_start_us":11662,"threads_started":1}
I20260811 03:16:12.904496 31714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.884474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:37377 (local address 127.0.0.1:43264)
0811 03:16:12.895415 (+ 10941us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:12.895433 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:12.895565 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:12.896331 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:12.896335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:12.896370 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:12.896813 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.896827 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.898653 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.898658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.901364 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.901379 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.903628 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.903634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.903830 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.903875 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 03:16:12.903966 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 03:16:12.904118 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10833,"mutex_wait_us":134,"thread_start_us":10693,"threads_started":1}
I20260811 03:16:12.904635 31711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.883470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:43321 (local address 127.0.0.1:55136)
0811 03:16:12.894424 (+ 10954us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:12.894450 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:12.894626 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:12.895524 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:12.895533 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:12.895578 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:12.896131 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.896148 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.898394 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.898401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.901364 (+ 2963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.901379 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.903812 (+ 2433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.903816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.904000 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.904042 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 03:16:12.904122 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 03:16:12.904246 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10752,"mutex_wait_us":260,"thread_start_us":10563,"threads_started":1}
I20260811 03:16:12.905061 31712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.884138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55136 (local address 127.11.60.3:43321)
0811 03:16:12.893824 (+ 9686us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:12.893835 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:12.894796 (+ 961us) server_negotiation.cc:407] Connection header received
0811 03:16:12.894933 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:12.894946 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:12.895281 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:12.895486 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:12.896478 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.898227 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.901623 (+ 3396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.903123 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.903258 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.904090 (+ 832us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:12.904172 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 03:16:12.904330 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9468,"thread_start_us":9275,"threads_started":1}
I20260811 03:16:12.905444 31715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.884906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.11.60.2:37377)
0811 03:16:12.893377 (+ 8471us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:12.893389 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:12.895722 (+ 2333us) server_negotiation.cc:407] Connection header received
0811 03:16:12.895826 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:12.895833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:12.896062 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:12.896335 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:12.897032 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.898492 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.901623 (+ 3131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.903124 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.903260 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.903489 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:12.904856 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0811 03:16:12.905053 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8282,"thread_start_us":8139,"threads_started":1}
I20260811 03:16:12.908185 31322 tablet_service.cc:1467] Processing CreateTablet for tablet e61170c7a20e48db8dbda39bbd3bb6f1 (DEFAULT_TABLE table=test-table [id=ab84d6bab51d422cb90ccb614341cdc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:12.908540 31622 tablet_service.cc:1467] Processing CreateTablet for tablet e61170c7a20e48db8dbda39bbd3bb6f1 (DEFAULT_TABLE table=test-table [id=ab84d6bab51d422cb90ccb614341cdc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:12.908934 31471 tablet_service.cc:1467] Processing CreateTablet for tablet e61170c7a20e48db8dbda39bbd3bb6f1 (DEFAULT_TABLE table=test-table [id=ab84d6bab51d422cb90ccb614341cdc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:12.909788 31322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61170c7a20e48db8dbda39bbd3bb6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:12.909845 31622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61170c7a20e48db8dbda39bbd3bb6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:12.910171 31471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61170c7a20e48db8dbda39bbd3bb6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:12.921196 31716 tablet_bootstrap.cc:492] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: Bootstrap starting.
I20260811 03:16:12.926383 31716 tablet_bootstrap.cc:654] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:12.927559 31716 log.cc:824] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:12.929047 31717 tablet_bootstrap.cc:492] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: Bootstrap starting.
I20260811 03:16:12.930429 31718 tablet_bootstrap.cc:492] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: Bootstrap starting.
I20260811 03:16:12.931080 31716 tablet_bootstrap.cc:492] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: No bootstrap required, opened a new log
I20260811 03:16:12.931363 31716 ts_tablet_manager.cc:1397] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 03:16:12.933564 31718 tablet_bootstrap.cc:654] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:12.934911 31718 log.cc:824] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:12.938374 31717 tablet_bootstrap.cc:654] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:12.939495 31717 log.cc:824] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:12.940039 31716 raft_consensus.cc:348] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } }
I20260811 03:16:12.940534 31716 raft_consensus.cc:374] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:12.940652 31716 raft_consensus.cc:729] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af4e9bfeb04942b587c5186641ad87ad, State: Initialized, Role: FOLLOWER
I20260811 03:16:12.941068 31718 tablet_bootstrap.cc:492] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: No bootstrap required, opened a new log
I20260811 03:16:12.941354 31718 ts_tablet_manager.cc:1397] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260811 03:16:12.941339 31716 consensus_queue.cc:260] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } }
I20260811 03:16:12.942487 31696 heartbeater.cc:503] Master 127.11.60.62:43109 was elected leader, sending a full tablet report...
I20260811 03:16:12.942742 31717 tablet_bootstrap.cc:492] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: No bootstrap required, opened a new log
I20260811 03:16:12.943004 31717 ts_tablet_manager.cc:1397] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260811 03:16:12.943198 31716 ts_tablet_manager.cc:1428] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 03:16:12.953992 31718 raft_consensus.cc:348] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } }
I20260811 03:16:12.953981 31717 raft_consensus.cc:348] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } }
I20260811 03:16:12.954531 31718 raft_consensus.cc:374] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:12.954530 31717 raft_consensus.cc:374] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:12.954648 31718 raft_consensus.cc:729] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 485c97cf3563432d8575a1c332a63478, State: Initialized, Role: FOLLOWER
I20260811 03:16:12.954649 31717 raft_consensus.cc:729] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 299f6970d13a452fa98652d19bbc0ad6, State: Initialized, Role: FOLLOWER
I20260811 03:16:12.955343 31718 consensus_queue.cc:260] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } }
I20260811 03:16:12.955343 31717 consensus_queue.cc:260] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } }
I20260811 03:16:12.957449 31717 ts_tablet_manager.cc:1428] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260811 03:16:12.957727 31718 ts_tablet_manager.cc:1428] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260811 03:16:12.957764 31545 heartbeater.cc:503] Master 127.11.60.62:43109 was elected leader, sending a full tablet report...
I20260811 03:16:12.963527 31724 raft_consensus.cc:482] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:16:12.963791 31724 raft_consensus.cc:504] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } }
I20260811 03:16:12.965320 31724 leader_election.cc:288] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 485c97cf3563432d8575a1c332a63478 (127.11.60.1:34421), af4e9bfeb04942b587c5186641ad87ad (127.11.60.3:43321)
I20260811 03:16:12.968647 31722 raft_consensus.cc:482] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:16:12.968986 31722 raft_consensus.cc:504] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } }
I20260811 03:16:12.971091 31722 leader_election.cc:288] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 485c97cf3563432d8575a1c332a63478 (127.11.60.1:34421), 299f6970d13a452fa98652d19bbc0ad6 (127.11.60.2:37377)
I20260811 03:16:12.975726 31712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.966387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:54127 (local address 127.11.60.3:43321)
0811 03:16:12.966661 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:12.966668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:12.967512 (+ 844us) server_negotiation.cc:407] Connection header received
0811 03:16:12.967776 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:12.967780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:12.967840 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:12.967941 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:12.968773 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.969766 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.973192 (+ 3426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.974549 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.974636 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.974752 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:12.975358 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 03:16:12.975485 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 03:16:12.976162 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.966230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:54537 (local address 127.11.60.1:34421)
0811 03:16:12.966429 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:12.966433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:12.967391 (+ 958us) server_negotiation.cc:407] Connection header received
0811 03:16:12.967754 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:12.967767 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:12.967857 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:12.967961 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:12.968826 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.969885 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.972869 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.974555 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.974687 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.974838 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:12.975584 (+ 746us) server_negotiation.cc:299] Negotiation successful
0811 03:16:12.975782 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 03:16:12.976498 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.965771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:43321 (local address 127.11.60.2:54127)
0811 03:16:12.967393 (+ 1622us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:12.967418 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:12.967571 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:12.967955 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:12.967959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:12.967991 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:12.968569 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.968589 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.969891 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.969899 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.972974 (+ 3075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.972988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.974948 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.974955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.975172 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.975205 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 03:16:12.975955 (+ 750us) client_negotiation.cc:240] Negotiation successful
0811 03:16:12.976131 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1502,"mutex_wait_us":179,"thread_start_us":1078,"threads_started":1}
I20260811 03:16:12.977501 31646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61170c7a20e48db8dbda39bbd3bb6f1" candidate_uuid: "299f6970d13a452fa98652d19bbc0ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4e9bfeb04942b587c5186641ad87ad" is_pre_election: true
I20260811 03:16:12.977985 31646 raft_consensus.cc:2454] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 299f6970d13a452fa98652d19bbc0ad6 in term 0.
I20260811 03:16:12.978786 31420 leader_election.cc:302] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 299f6970d13a452fa98652d19bbc0ad6, af4e9bfeb04942b587c5186641ad87ad; no voters:
I20260811 03:16:12.979249 31724 raft_consensus.cc:2781] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:16:12.979431 31724 raft_consensus.cc:482] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:16:12.979557 31724 raft_consensus.cc:3037] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:12.981034 31715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.972212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:60415 (local address 127.11.60.2:37377)
0811 03:16:12.972381 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:12.972386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:12.972405 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:16:12.972462 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:12.972469 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:12.972583 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:12.972674 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:12.974622 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.975738 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.978721 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.979874 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.979952 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.980010 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:12.980583 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 03:16:12.980733 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 03:16:12.980865 31727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.965613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34421 (local address 127.11.60.2:54537)
0811 03:16:12.967267 (+ 1654us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:12.967289 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:12.967422 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:12.967976 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:12.967981 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:12.968005 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:12.968672 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.968705 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.970128 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.970134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.972650 (+ 2516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.972664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.975092 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.975105 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.975352 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.975403 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 03:16:12.977111 (+ 1708us) client_negotiation.cc:240] Negotiation successful
0811 03:16:12.980429 (+ 3318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1458,"mutex_wait_us":49,"spinlock_wait_cycles":33152,"thread_start_us":1438,"threads_started":1}
I20260811 03:16:12.981415 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.971641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:37377 (local address 127.11.60.3:60415)
0811 03:16:12.972197 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:12.972216 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:12.972311 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:12.973861 (+ 1550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:12.973866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:12.973892 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:12.974367 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.974379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.975974 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.975979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.978496 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.978512 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.980139 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.980143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.980386 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.980422 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 03:16:12.980902 (+ 480us) client_negotiation.cc:240] Negotiation successful
0811 03:16:12.981031 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":29,"thread_start_us":2033,"threads_started":1}
I20260811 03:16:12.982415 31344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61170c7a20e48db8dbda39bbd3bb6f1" candidate_uuid: "299f6970d13a452fa98652d19bbc0ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "485c97cf3563432d8575a1c332a63478" is_pre_election: true
I20260811 03:16:12.982829 31724 raft_consensus.cc:504] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } }
I20260811 03:16:12.983121 31344 raft_consensus.cc:2454] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 299f6970d13a452fa98652d19bbc0ad6 in term 0.
I20260811 03:16:12.984007 31646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61170c7a20e48db8dbda39bbd3bb6f1" candidate_uuid: "299f6970d13a452fa98652d19bbc0ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4e9bfeb04942b587c5186641ad87ad"
I20260811 03:16:12.983933 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61170c7a20e48db8dbda39bbd3bb6f1" candidate_uuid: "af4e9bfeb04942b587c5186641ad87ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "299f6970d13a452fa98652d19bbc0ad6" is_pre_election: true
I20260811 03:16:12.984222 31646 raft_consensus.cc:3037] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:12.984426 31495 raft_consensus.cc:2379] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate af4e9bfeb04942b587c5186641ad87ad in current term 1: Already voted for candidate 299f6970d13a452fa98652d19bbc0ad6 in this term.
I20260811 03:16:12.984519 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.973101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:47287 (local address 127.11.60.1:34421)
0811 03:16:12.976447 (+ 3346us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:12.976454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:12.976475 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:16:12.976543 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:12.976550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:12.976645 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:12.976745 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:12.977171 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.979063 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.981761 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.983156 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.983292 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.983389 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:12.984101 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 03:16:12.984238 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3215,"threads_started":1}
I20260811 03:16:12.985073 31344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61170c7a20e48db8dbda39bbd3bb6f1" candidate_uuid: "299f6970d13a452fa98652d19bbc0ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "485c97cf3563432d8575a1c332a63478"
I20260811 03:16:12.985289 31344 raft_consensus.cc:3037] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:12.986644 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:12.971286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:34421 (local address 127.11.60.3:47287)
0811 03:16:12.971438 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:12.971456 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:12.971561 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:12.976772 (+ 5211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:12.976777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:12.976802 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:12.977028 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.977039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.979266 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.979274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.981216 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:12.981229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:12.983485 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:12.983490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:12.983690 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:12.983722 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 03:16:12.983823 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 03:16:12.983979 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":51}
I20260811 03:16:12.987170 31724 leader_election.cc:288] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 485c97cf3563432d8575a1c332a63478 (127.11.60.1:34421), af4e9bfeb04942b587c5186641ad87ad (127.11.60.3:43321)
I20260811 03:16:12.987267 31646 raft_consensus.cc:2454] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 299f6970d13a452fa98652d19bbc0ad6 in term 1.
I20260811 03:16:12.987771 31420 leader_election.cc:302] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 299f6970d13a452fa98652d19bbc0ad6, af4e9bfeb04942b587c5186641ad87ad; no voters:
I20260811 03:16:12.987886 31344 raft_consensus.cc:2454] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 299f6970d13a452fa98652d19bbc0ad6 in term 1.
I20260811 03:16:12.988245 31724 raft_consensus.cc:2781] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:12.988744 31724 raft_consensus.cc:686] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 299f6970d13a452fa98652d19bbc0ad6, State: Running, Role: LEADER
I20260811 03:16:12.988916 31344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61170c7a20e48db8dbda39bbd3bb6f1" candidate_uuid: "af4e9bfeb04942b587c5186641ad87ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "485c97cf3563432d8575a1c332a63478" is_pre_election: true
I20260811 03:16:12.989151 31344 raft_consensus.cc:2379] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate af4e9bfeb04942b587c5186641ad87ad in current term 1: Already voted for candidate 299f6970d13a452fa98652d19bbc0ad6 in this term.
I20260811 03:16:12.989310 31724 consensus_queue.cc:237] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } }
I20260811 03:16:12.989647 31569 leader_election.cc:302] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: af4e9bfeb04942b587c5186641ad87ad; no voters: 299f6970d13a452fa98652d19bbc0ad6, 485c97cf3563432d8575a1c332a63478
I20260811 03:16:12.990072 31722 raft_consensus.cc:2726] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 03:16:12.994359 31176 catalog_manager.cc:5495] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 299f6970d13a452fa98652d19bbc0ad6 (127.11.60.2). New cstate: current_term: 1 leader_uuid: "299f6970d13a452fa98652d19bbc0ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "299f6970d13a452fa98652d19bbc0ad6" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 37377 } health_report { overall_health: HEALTHY } } }
I20260811 03:16:13.030997 31715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:13.024598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.11.60.2:37377)
0811 03:16:13.024824 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:13.024830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:13.024857 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:16:13.025104 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:13.025108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:13.025217 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:13.025385 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:16:13.025822 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:13.026860 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:13.028341 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:13.028886 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:13.028996 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:13.029707 (+ 711us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:16:13.030405 (+ 698us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:16:13.030539 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:13.030684 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 03:16:13.030755 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 03:16:13.039611 31646 raft_consensus.cc:1261] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 1 FOLLOWER]: Refusing update from remote peer 299f6970d13a452fa98652d19bbc0ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:13.039621 31344 raft_consensus.cc:1261] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [term 1 FOLLOWER]: Refusing update from remote peer 299f6970d13a452fa98652d19bbc0ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:13.040565 31732 consensus_queue.cc:1035] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "485c97cf3563432d8575a1c332a63478" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 34421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:13.041052 31724 consensus_queue.cc:1035] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af4e9bfeb04942b587c5186641ad87ad" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:13.052896 31736 mvcc.cc:205] Tried to move back new op lower bound from 7317168836760461312 to 7317168836569399296. Current Snapshot: MvccSnapshot[applied={T|T < 7317168836760461312}]
I20260811 03:16:13.056586 31735 mvcc.cc:205] Tried to move back new op lower bound from 7317168836760461312 to 7317168836569399296. Current Snapshot: MvccSnapshot[applied={T|T < 7317168836760461312}]
I20260811 03:16:13.068907 31176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:13.069182 31176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:13.072988 31176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e410a7287eb45feb8449d940457d95c: Sending DeleteTablet for 3 replicas of tablet e61170c7a20e48db8dbda39bbd3bb6f1
I20260811 03:16:13.074220 31622 tablet_service.cc:1514] Processing DeleteTablet for tablet e61170c7a20e48db8dbda39bbd3bb6f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:13 UTC) from {username='slave'} at 127.0.0.1:55136
I20260811 03:16:13.074424 31322 tablet_service.cc:1514] Processing DeleteTablet for tablet e61170c7a20e48db8dbda39bbd3bb6f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:13 UTC) from {username='slave'} at 127.0.0.1:57466
I20260811 03:16:13.075197 31471 tablet_service.cc:1514] Processing DeleteTablet for tablet e61170c7a20e48db8dbda39bbd3bb6f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:13 UTC) from {username='slave'} at 127.0.0.1:43264
I20260811 03:16:13.076115 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 31250
I20260811 03:16:13.076138 31746 tablet_replica.cc:330] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: stopping tablet replica
I20260811 03:16:13.076320 31747 tablet_replica.cc:330] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: stopping tablet replica
I20260811 03:16:13.076799 31747 raft_consensus.cc:2229] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:16:13.076895 31746 raft_consensus.cc:2229] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:13.077517 31747 raft_consensus.cc:2258] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:13.077606 31746 raft_consensus.cc:2258] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:13.078606 31745 tablet_replica.cc:330] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: stopping tablet replica
I20260811 03:16:13.079105 31745 raft_consensus.cc:2229] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:13.079846 31745 raft_consensus.cc:2258] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:13.080453 31746 ts_tablet_manager.cc:1905] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:13.080886 31747 ts_tablet_manager.cc:1905] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:13.086596 31745 ts_tablet_manager.cc:1905] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:13.088712 31746 ts_tablet_manager.cc:1918] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:13.088925 31746 log.cc:1196] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/e61170c7a20e48db8dbda39bbd3bb6f1
I20260811 03:16:13.089550 31746 ts_tablet_manager.cc:1939] T e61170c7a20e48db8dbda39bbd3bb6f1 P 485c97cf3563432d8575a1c332a63478: Deleting consensus metadata
I20260811 03:16:13.091583 31160 catalog_manager.cc:4841] TS 485c97cf3563432d8575a1c332a63478 (127.11.60.1:34421): tablet e61170c7a20e48db8dbda39bbd3bb6f1 (table test-table [id=ab84d6bab51d422cb90ccb614341cdc8]) successfully deleted
I20260811 03:16:13.093638 31747 ts_tablet_manager.cc:1918] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:13.093703 31745 ts_tablet_manager.cc:1918] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:13.093823 31747 log.cc:1196] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/e61170c7a20e48db8dbda39bbd3bb6f1
I20260811 03:16:13.094523 31747 ts_tablet_manager.cc:1939] T e61170c7a20e48db8dbda39bbd3bb6f1 P 299f6970d13a452fa98652d19bbc0ad6: Deleting consensus metadata
I20260811 03:16:13.094550 31745 log.cc:1196] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/e61170c7a20e48db8dbda39bbd3bb6f1
I20260811 03:16:13.095108 31745 ts_tablet_manager.cc:1939] T e61170c7a20e48db8dbda39bbd3bb6f1 P af4e9bfeb04942b587c5186641ad87ad: Deleting consensus metadata
I20260811 03:16:13.096487 31160 catalog_manager.cc:4841] TS 299f6970d13a452fa98652d19bbc0ad6 (127.11.60.2:37377): tablet e61170c7a20e48db8dbda39bbd3bb6f1 (table test-table [id=ab84d6bab51d422cb90ccb614341cdc8]) successfully deleted
I20260811 03:16:13.096877 31729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:13.077354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57480 (local address 127.11.60.1:34421)
0811 03:16:13.077546 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:13.077553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:13.087350 (+ 9797us) server_negotiation.cc:407] Connection header received
0811 03:16:13.087590 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:13.087595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:13.087699 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:13.087807 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:13.088661 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:13.089984 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:13.092135 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:13.092822 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:13.092962 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:13.094631 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:13.094648 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:13.094671 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:13.095578 (+ 907us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:13.095645 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:13.095683 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:13.095771 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:13.096244 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 03:16:13.096360 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 03:16:13.096923 31158 catalog_manager.cc:4841] TS af4e9bfeb04942b587c5186641ad87ad (127.11.60.3:43321): tablet e61170c7a20e48db8dbda39bbd3bb6f1 (table test-table [id=ab84d6bab51d422cb90ccb614341cdc8]) successfully deleted
I20260811 03:16:13.098150 31391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:13.190587 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 31250
I20260811 03:16:13.220571 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 31401
I20260811 03:16:13.229066 31715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:13.221398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.11.60.2:37377)
0811 03:16:13.221602 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:13.221609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:13.221632 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:16:13.221765 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:13.221771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:13.221913 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:13.222036 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:13.222561 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:13.223773 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:13.225242 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:13.225949 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:13.226082 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:13.227477 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:13.227486 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:13.227505 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:13.228186 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:13.228236 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:13.228260 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:13.228393 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:13.228545 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 03:16:13.228667 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:16:13.229820 31538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:13.348299 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 31401
I20260811 03:16:13.370862 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 31549
I20260811 03:16:13.377910 31712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:13.371762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55140 (local address 127.11.60.3:43321)
0811 03:16:13.371943 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:13.371947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:13.371965 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:16:13.372081 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:13.372085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:13.372168 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:13.372256 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:13.372637 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:13.373401 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:13.374890 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:13.375304 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:13.375411 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:13.376625 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:13.376634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:13.376649 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:13.377257 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:13.377304 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:13.377324 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:13.377413 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:13.377564 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 03:16:13.377657 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 03:16:13.378616 31689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:13.488021 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 31549
I20260811 03:16:13.507368 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 31135
I20260811 03:16:13.508652 31214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:13.620879 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 31135
2026-08-11T03:16:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2890 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:13 dist-test-slave-tm1g krb5kdc[31756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:13 dist-test-slave-tm1g krb5kdc[31756](info): set up 2 sockets
Aug 11 03:16:13 dist-test-slave-tm1g krb5kdc[31756](info): commencing operation
krb5kdc: starting...
W20260811 03:16:14.818020 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:15 dist-test-slave-tm1g krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:16:15.134914 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:36197
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:36197
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:15.483860 31772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:15.484225 31772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:15.484390 31772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:15.492017 31772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:15.492199 31772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:15.492317 31772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:15.492434 31772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:15.492524 31772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:15.502590 31772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:36197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:36197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:15.504058 31772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:15.505452 31772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:15.517840 31779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:15.517930 31777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:15.517863 31782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:15.518846 31772 server_base.cc:1034] running on GCE node
I20260811 03:16:15.519610 31772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:15.521250 31772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:15.522481 31772 hybrid_clock.cc:648] HybridClock initialized: now 1786418175522404 us; error 42 us; skew 500 ppm
I20260811 03:16:15.522976 31772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:15 dist-test-slave-tm1g krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:15.526247 31772 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:16:15.527860 31772 webserver.cc:460] Webserver started at http://127.11.60.62:35645/ using document root <none> and password file <none>
I20260811 03:16:15.528587 31772 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:15.528738 31772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:15.529048 31772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:15.532202 31772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "82b431d9c4a0488d9b7017d760c15fc1"
format_stamp: "Formatted at 2026-08-11 03:16:15 on dist-test-slave-tm1g"
server_key: "7a5af1a33b02df1fb1212c4205ec79c8"
server_key_iv: "b8c88dc0be7aadd2ba2d5b7162436ae4"
server_key_version: "encryptionkey@0"
I20260811 03:16:15.533005 31772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "82b431d9c4a0488d9b7017d760c15fc1"
format_stamp: "Formatted at 2026-08-11 03:16:15 on dist-test-slave-tm1g"
server_key: "7a5af1a33b02df1fb1212c4205ec79c8"
server_key_iv: "b8c88dc0be7aadd2ba2d5b7162436ae4"
server_key_version: "encryptionkey@0"
I20260811 03:16:15.538199 31772 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 03:16:15.541712 31790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:15.543093 31772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:16:15.543309 31772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "82b431d9c4a0488d9b7017d760c15fc1"
format_stamp: "Formatted at 2026-08-11 03:16:15 on dist-test-slave-tm1g"
server_key: "7a5af1a33b02df1fb1212c4205ec79c8"
server_key_iv: "b8c88dc0be7aadd2ba2d5b7162436ae4"
server_key_version: "encryptionkey@0"
I20260811 03:16:15.543517 31772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:15.571766 31772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:15.576359 31772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:15.576722 31772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:15.596203 31772 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:36197
I20260811 03:16:15.596252 31846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:36197 every 8 connection(s)
I20260811 03:16:15.597657 31772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:15.601583 31847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:15.605712 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 31772
I20260811 03:16:15.605914 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:15.606516 11504 external_mini_cluster.cc:1428] Setting key 5070db891128f5359b0b06682fc653e2
I20260811 03:16:15.615304 31847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1: Bootstrap starting.
I20260811 03:16:15.619284 31847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:15.620745 31847 log.cc:824] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1: Log is configured to *not* fsync() on all Append() calls
Aug 11 03:16:15 dist-test-slave-tm1g krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:15.624105 31847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1: No bootstrap required, opened a new log
I20260811 03:16:15.631318 31850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:15.608477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.11.60.62:36197)
0811 03:16:15.609268 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:15.609280 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:15.609320 (+ 40us) server_negotiation.cc:407] Connection header received
0811 03:16:15.611028 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:15.611050 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:15.611458 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:15.611787 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:15.613707 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:15.615068 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:15.616980 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:15.617664 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:15.617798 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:15.623327 (+ 5529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:15.623336 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:15.623350 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:15.623391 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:15.626800 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:15.627611 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:15.627624 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:15.627630 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:15.627810 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:15.628220 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:15.628225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:15.628230 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:15.628826 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:15.629150 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:15.629649 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 03:16:15.630108 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":384,"threads_started":1}
I20260811 03:16:15.632820 31847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b431d9c4a0488d9b7017d760c15fc1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36197 } }
I20260811 03:16:15.633302 31847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:15.633412 31847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b431d9c4a0488d9b7017d760c15fc1, State: Initialized, Role: FOLLOWER
I20260811 03:16:15.634056 31847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b431d9c4a0488d9b7017d760c15fc1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36197 } }
I20260811 03:16:15.634400 31847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:15.634549 31847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:15.634708 31847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:15.636893 31847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b431d9c4a0488d9b7017d760c15fc1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36197 } }
I20260811 03:16:15.637493 31847 leader_election.cc:302] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82b431d9c4a0488d9b7017d760c15fc1; no voters:
I20260811 03:16:15.638146 31847 leader_election.cc:288] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:15.638388 31852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:15.639109 31852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [term 1 LEADER]: Becoming Leader. State: Replica: 82b431d9c4a0488d9b7017d760c15fc1, State: Running, Role: LEADER
I20260811 03:16:15.639662 31852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b431d9c4a0488d9b7017d760c15fc1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36197 } }
I20260811 03:16:15.640214 31847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:15.642693 31853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82b431d9c4a0488d9b7017d760c15fc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b431d9c4a0488d9b7017d760c15fc1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36197 } } }
I20260811 03:16:15.643121 31853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:15.643738 31854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82b431d9c4a0488d9b7017d760c15fc1. Latest consensus state: current_term: 1 leader_uuid: "82b431d9c4a0488d9b7017d760c15fc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b431d9c4a0488d9b7017d760c15fc1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 36197 } } }
I20260811 03:16:15.644009 31854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:15.646037 31862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:15.650398 31862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:15.658701 31862 catalog_manager.cc:1347] Generated new cluster ID: 6425556dbe1b40059d1decc8e4dc71ac
I20260811 03:16:15.658833 31862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:15.669683 31862 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:15.671634 31862 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:15.681311 31862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1: Generated new TSK 0
I20260811 03:16:15.682461 31862 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:16:15.774816 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36197
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:16.133378 31876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:16.133702 31876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:16.133870 31876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:16.141429 31876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:16.141571 31876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:16.141757 31876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:16:16.152339 31876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:16.153674 31876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:16.155126 31876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:16.169229 31884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:16.169181 31882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:16.169283 31881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:16.175653 31876 server_base.cc:1034] running on GCE node
I20260811 03:16:16.176338 31876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:16.177333 31876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:16.178614 31876 hybrid_clock.cc:648] HybridClock initialized: now 1786418176178568 us; error 41 us; skew 500 ppm
I20260811 03:16:16.178944 31876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:16 dist-test-slave-tm1g krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418176, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:16.182485 31876 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:16:16.184127 31876 webserver.cc:460] Webserver started at http://127.11.60.1:39839/ using document root <none> and password file <none>
I20260811 03:16:16.184886 31876 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:16.185030 31876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:16.185374 31876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:16.188586 31876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "d2944e5a53c040b08f60f646b744f8a6"
format_stamp: "Formatted at 2026-08-11 03:16:16 on dist-test-slave-tm1g"
server_key: "6fe87c72ad5031487ed9d46dc06fb081"
server_key_iv: "fa8c4c57f67b568081bcf74afcbe79b5"
server_key_version: "encryptionkey@0"
I20260811 03:16:16.189412 31876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "d2944e5a53c040b08f60f646b744f8a6"
format_stamp: "Formatted at 2026-08-11 03:16:16 on dist-test-slave-tm1g"
server_key: "6fe87c72ad5031487ed9d46dc06fb081"
server_key_iv: "fa8c4c57f67b568081bcf74afcbe79b5"
server_key_version: "encryptionkey@0"
I20260811 03:16:16.194957 31876 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 03:16:16.198560 31891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:16.199878 31876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:16:16.200054 31876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "d2944e5a53c040b08f60f646b744f8a6"
format_stamp: "Formatted at 2026-08-11 03:16:16 on dist-test-slave-tm1g"
server_key: "6fe87c72ad5031487ed9d46dc06fb081"
server_key_iv: "fa8c4c57f67b568081bcf74afcbe79b5"
server_key_version: "encryptionkey@0"
I20260811 03:16:16.200269 31876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:16.225876 31876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:16.230708 31876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:16.231040 31876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:16.232105 31876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:16.233584 31876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:16.233698 31876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:16.233862 31876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:16.233938 31876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:16.269093 31876 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:40093
I20260811 03:16:16.269186 32022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:40093 every 8 connection(s)
I20260811 03:16:16.270546 31876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:16:16.278893 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 31876
I20260811 03:16:16.279115 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:16:16.279822 11504 external_mini_cluster.cc:1428] Setting key 45c25658877a1b6254f3fe47ea459aab
Aug 11 03:16:16 dist-test-slave-tm1g krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418176, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:16.295092 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:16.273235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:44973 (local address 127.11.60.62:36197)
0811 03:16:16.273733 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:16.273738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:16.274887 (+ 1149us) server_negotiation.cc:407] Connection header received
0811 03:16:16.276912 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:16.276916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:16.277030 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:16.277191 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:16.279887 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:16.281184 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:16.283953 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:16.284373 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:16.284529 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:16.289252 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:16.289258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:16.289261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:16.289309 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:16.291912 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:16.292655 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:16.292661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:16.292664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:16.292795 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:16.293238 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:16.293243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:16.293247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:16.293551 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:16.293718 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:16.294463 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 03:16:16.294695 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":273,"threads_started":1}
I20260811 03:16:16.296051 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:16.273834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:36197 (local address 127.11.60.1:44973)
0811 03:16:16.274690 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:16.274733 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:16.276636 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:16.277444 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:16.277452 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:16.278404 (+ 952us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:16.279650 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:16.279684 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:16.281367 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:16.281373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:16.283764 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:16.283780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:16.284620 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:16.284629 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:16.284951 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:16.285805 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:16:16.285848 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:16.289005 (+ 3157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:16.292057 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:16.292066 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:16.292083 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:16.292525 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:16.292908 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:16.292915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:16.292919 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:16.293116 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:16.293694 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:16.293702 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:16.294055 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 03:16:16.294339 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 03:16:16.294894 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":316,"threads_started":1}
I20260811 03:16:16.297804 32023 heartbeater.cc:347] Connected to a master server at 127.11.60.62:36197
I20260811 03:16:16.298132 32023 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:16.298919 32023 heartbeater.cc:511] Master 127.11.60.62:36197 requested a full tablet report, sending...
I20260811 03:16:16.301282 31808 ts_manager.cc:194] Registered new tserver with Master: d2944e5a53c040b08f60f646b744f8a6 (127.11.60.1:40093)
I20260811 03:16:16.303646 31808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:44973
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:16:16.358242 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36197
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:16.703660 32029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:16.704007 32029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:16.704181 32029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:16.711691 32029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:16.711850 32029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:16.712046 32029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:16:16.722004 32029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:16.723420 32029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:16.724859 32029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:16.738860 32036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:16.739224 32037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:16.739252 32039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:16.740278 32029 server_base.cc:1034] running on GCE node
I20260811 03:16:16.740923 32029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:16.741889 32029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:16.743131 32029 hybrid_clock.cc:648] HybridClock initialized: now 1786418176743092 us; error 49 us; skew 500 ppm
I20260811 03:16:16.743496 32029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:16 dist-test-slave-tm1g krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418176, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:16.746766 32029 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:16:16.748238 32029 webserver.cc:460] Webserver started at http://127.11.60.2:36007/ using document root <none> and password file <none>
I20260811 03:16:16.748911 32029 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:16.749015 32029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:16.749290 32029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:16.752218 32029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "ad9fc0bb73884deebe943536ffc685f1"
format_stamp: "Formatted at 2026-08-11 03:16:16 on dist-test-slave-tm1g"
server_key: "2f7311353e296ada378a56e1835caf37"
server_key_iv: "7f57320967421f97535fe8d1c0bb1a3f"
server_key_version: "encryptionkey@0"
I20260811 03:16:16.752928 32029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "ad9fc0bb73884deebe943536ffc685f1"
format_stamp: "Formatted at 2026-08-11 03:16:16 on dist-test-slave-tm1g"
server_key: "2f7311353e296ada378a56e1835caf37"
server_key_iv: "7f57320967421f97535fe8d1c0bb1a3f"
server_key_version: "encryptionkey@0"
I20260811 03:16:16.758073 32029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 03:16:16.761610 32046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:16.763005 32029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:16:16.763147 32029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "ad9fc0bb73884deebe943536ffc685f1"
format_stamp: "Formatted at 2026-08-11 03:16:16 on dist-test-slave-tm1g"
server_key: "2f7311353e296ada378a56e1835caf37"
server_key_iv: "7f57320967421f97535fe8d1c0bb1a3f"
server_key_version: "encryptionkey@0"
I20260811 03:16:16.763314 32029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:16.781347 32029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:16.785418 32029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:16.785660 32029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:16.786584 32029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:16.787880 32029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:16.787956 32029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:16.788089 32029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:16.788146 32029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:16.823287 32029 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:46231
I20260811 03:16:16.823335 32174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:46231 every 8 connection(s)
I20260811 03:16:16.824643 32029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:16:16.829129 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 32029
I20260811 03:16:16.829349 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:16:16.829808 11504 external_mini_cluster.cc:1428] Setting key 05593b1f140340f01da07ccba976851d
Aug 11 03:16:16 dist-test-slave-tm1g krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418176, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:16.847954 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:16.827377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:33433 (local address 127.11.60.62:36197)
0811 03:16:16.827842 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:16.827846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:16.828668 (+ 822us) server_negotiation.cc:407] Connection header received
0811 03:16:16.830858 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:16.830864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:16.830958 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:16.831078 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:16.832767 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:16.833489 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:16.835253 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:16.835703 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:16.835844 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:16.839443 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:16.839454 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:16.839457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:16.839491 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:16.843734 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:16.844865 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:16.844871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:16.844874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:16.845104 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:16.845713 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:16.845720 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:16.845725 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:16.846099 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:16.846249 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:16.847254 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0811 03:16:16.847547 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":361,"thread_start_us":253,"threads_started":1}
I20260811 03:16:16.849164 32179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:16.827768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:36197 (local address 127.11.60.2:33433)
0811 03:16:16.828511 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:16.828549 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:16.830561 (+ 2012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:16.831336 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:16.831345 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:16.832025 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:16.832595 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:16.832613 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:16.833703 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:16.833707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:16.835101 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:16.835114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:16.835952 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:16.835960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:16.836187 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:16.836911 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:16:16.836940 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:16.839188 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:16.843931 (+ 4743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:16.843940 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:16.843957 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:16.844716 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:16.845209 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:16.845215 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:16.845218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:16.845553 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:16.846245 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:16.846256 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:16.846788 (+ 532us) client_negotiation.cc:769] Sending connection context
0811 03:16:16.847110 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 03:16:16.847703 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":292,"threads_started":1}
I20260811 03:16:16.850858 32175 heartbeater.cc:347] Connected to a master server at 127.11.60.62:36197
I20260811 03:16:16.851290 32175 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:16.852221 32175 heartbeater.cc:511] Master 127.11.60.62:36197 requested a full tablet report, sending...
I20260811 03:16:16.853783 31808 ts_manager.cc:194] Registered new tserver with Master: ad9fc0bb73884deebe943536ffc685f1 (127.11.60.2:46231)
I20260811 03:16:16.854691 31808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:33433
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:16:16.903115 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36197
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:17.244189 32183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:17.244516 32183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:17.244694 32183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:17.252034 32183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:17.252182 32183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:17.252375 32183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:16:17.262243 32183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:36197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:17.263621 32183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:17.264953 32183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:17.279660 32191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:17.279716 32189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:17.279686 32188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:17.280531 32183 server_base.cc:1034] running on GCE node
I20260811 03:16:17.281087 32183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:17.282007 32183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:17.283236 32183 hybrid_clock.cc:648] HybridClock initialized: now 1786418177283147 us; error 78 us; skew 500 ppm
I20260811 03:16:17.283607 32183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:17 dist-test-slave-tm1g krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418177, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:17.287065 32183 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:16:17.288666 32183 webserver.cc:460] Webserver started at http://127.11.60.3:40245/ using document root <none> and password file <none>
I20260811 03:16:17.289408 32183 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:17.289563 32183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:17.289901 32183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:17.293172 32183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "488bda0764eb495fb7f26ada67276ecb"
format_stamp: "Formatted at 2026-08-11 03:16:17 on dist-test-slave-tm1g"
server_key: "bfdf3b17d5fcdf20a0283ea5d64dc093"
server_key_iv: "a70dcb161542ac1e1fbc390dd5a7205a"
server_key_version: "encryptionkey@0"
I20260811 03:16:17.294031 32183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "488bda0764eb495fb7f26ada67276ecb"
format_stamp: "Formatted at 2026-08-11 03:16:17 on dist-test-slave-tm1g"
server_key: "bfdf3b17d5fcdf20a0283ea5d64dc093"
server_key_iv: "a70dcb161542ac1e1fbc390dd5a7205a"
server_key_version: "encryptionkey@0"
I20260811 03:16:17.299587 32183 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 03:16:17.303530 32198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:17.305470 32183 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 03:16:17.305655 32183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "488bda0764eb495fb7f26ada67276ecb"
format_stamp: "Formatted at 2026-08-11 03:16:17 on dist-test-slave-tm1g"
server_key: "bfdf3b17d5fcdf20a0283ea5d64dc093"
server_key_iv: "a70dcb161542ac1e1fbc390dd5a7205a"
server_key_version: "encryptionkey@0"
I20260811 03:16:17.305866 32183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:17.306981 32023 heartbeater.cc:503] Master 127.11.60.62:36197 was elected leader, sending a full tablet report...
I20260811 03:16:17.332696 32183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:17.337482 32183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:17.337733 32183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:17.338763 32183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:17.340131 32183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:17.340204 32183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:17.340294 32183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:17.340338 32183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:17.374715 32183 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:42783
I20260811 03:16:17.374755 32322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:42783 every 8 connection(s)
I20260811 03:16:17.376142 32183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:16:17.384450 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 32183
I20260811 03:16:17.384660 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:16:17.385138 11504 external_mini_cluster.cc:1428] Setting key 95f5113dffd6f50a8a02148ffc67eab9
Aug 11 03:16:17 dist-test-slave-tm1g krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418177, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:17.396507 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.378992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:34949 (local address 127.11.60.62:36197)
0811 03:16:17.379538 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:17.379543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:17.380501 (+ 958us) server_negotiation.cc:407] Connection header received
0811 03:16:17.382299 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:17.382351 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:17.382436 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:17.382546 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:17.384050 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.384986 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.386911 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.387364 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.387505 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.390910 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:17.390915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:17.390919 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:17.390952 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:17.393437 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:17.394152 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:17.394159 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:17.394162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:17.394288 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:17.394758 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:17.394764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:17.394767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:17.395041 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:17.395198 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:17.395875 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 03:16:17.396101 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":240,"threads_started":1}
I20260811 03:16:17.397418 32326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.379536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:36197 (local address 127.11.60.3:34949)
0811 03:16:17.380296 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:17.380333 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:17.382025 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:17.382754 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:17.382762 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:17.383286 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:17.383870 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.383888 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.385138 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.385150 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.386762 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.386774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.387605 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.387615 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.387841 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.388527 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:16:17.388548 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:17.390706 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:17.393576 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:17.393584 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:17.393600 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:17.394032 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:17.394454 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:17.394461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:17.394464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:17.394645 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:17.395179 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:17.395187 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:17.395511 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 03:16:17.395781 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 03:16:17.396312 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":322,"threads_started":1}
I20260811 03:16:17.398891 32323 heartbeater.cc:347] Connected to a master server at 127.11.60.62:36197
I20260811 03:16:17.399221 32323 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:17.400074 32323 heartbeater.cc:511] Master 127.11.60.62:36197 requested a full tablet report, sending...
I20260811 03:16:17.401537 31808 ts_manager.cc:194] Registered new tserver with Master: 488bda0764eb495fb7f26ada67276ecb (127.11.60.3:42783)
I20260811 03:16:17.402365 31808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:34949
I20260811 03:16:17.412854 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:16:17.430824 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.417169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.11.60.62:36197)
0811 03:16:17.417406 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:17.417410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:17.417836 (+ 426us) server_negotiation.cc:407] Connection header received
0811 03:16:17.417981 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:17.417985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:17.418051 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:17.418142 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:17.419670 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.420476 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.421843 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.422275 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.422485 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.425261 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:17.425266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:17.425270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:17.425303 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:17.427947 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:17.428756 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:17.428759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:17.428761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:17.428882 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:17.429346 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:17.429349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:17.429351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:17.429662 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:17.429862 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:17.430218 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 03:16:17.430570 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 03:16:17.435070 31808 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:16:17.442184 31808 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:16:17.476737 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.455248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:42783 (local address 127.0.0.1:36848)
0811 03:16:17.465617 (+ 10369us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:17.465689 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:17.466653 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:17.468030 (+ 1377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:17.468043 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:17.468091 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:17.468728 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.468749 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.471087 (+ 2338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.471095 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.473050 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.473066 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.475612 (+ 2546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.475622 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.475912 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.475977 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 03:16:17.476126 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 03:16:17.476325 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9468,"thread_start_us":14021,"threads_started":1}
I20260811 03:16:17.476758 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.455840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36848 (local address 127.11.60.3:42783)
0811 03:16:17.466820 (+ 10980us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:17.466834 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:17.466877 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:16:17.467005 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:17.467016 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:17.467314 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:17.467531 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:17.468958 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.470885 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.473294 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.475027 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.475176 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.475429 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:17.476194 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 03:16:17.476388 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10783,"thread_start_us":10616,"threads_started":1}
I20260811 03:16:17.478422 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.455446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.11.60.1:40093)
0811 03:16:17.468426 (+ 12980us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:17.468436 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:17.469911 (+ 1475us) server_negotiation.cc:407] Connection header received
0811 03:16:17.470023 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:17.470031 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:17.470259 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:17.470505 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:17.471241 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.472841 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.475841 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.477202 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.477361 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.477699 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:17.477899 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 03:16:17.478054 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12705,"thread_start_us":13465,"threads_started":1}
I20260811 03:16:17.479346 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.457388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.11.60.2:46231)
0811 03:16:17.467881 (+ 10493us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:17.467894 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:17.470015 (+ 2121us) server_negotiation.cc:407] Connection header received
0811 03:16:17.470356 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:17.470366 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:17.470589 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:17.470808 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:17.471605 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.473081 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.475926 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.477581 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.477712 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.477925 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:17.478803 (+ 878us) server_negotiation.cc:299] Negotiation successful
0811 03:16:17.478996 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10264,"thread_start_us":10148,"threads_started":1}
I20260811 03:16:17.480042 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.457166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:46231 (local address 127.0.0.1:34976)
0811 03:16:17.469894 (+ 12728us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:17.469912 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:17.470128 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:17.470834 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:17.470842 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:17.470877 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:17.471413 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.471428 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.473274 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.473280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.475678 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.475690 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.478084 (+ 2394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.478097 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.478282 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.478346 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 03:16:17.478466 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 03:16:17.479549 (+ 1083us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12609,"mutex_wait_us":20,"thread_start_us":7349,"threads_started":1}
I20260811 03:16:17.480324 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.455321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:40093 (local address 127.0.0.1:48766)
0811 03:16:17.469602 (+ 14281us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:17.469623 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:17.469760 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:17.470488 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:17.470493 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:17.470519 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:17.471031 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.471044 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.473012 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.473018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.475654 (+ 2636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.475668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.477483 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.477491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.477692 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.477728 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 03:16:17.477822 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 03:16:17.477959 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14092,"mutex_wait_us":33,"thread_start_us":13252,"threads_started":1}
I20260811 03:16:17.481911 32252 tablet_service.cc:1467] Processing CreateTablet for tablet 934c68ab24ae453c8c373e811bc1e9eb (DEFAULT_TABLE table=test-table [id=166229d4910f4369a4293898daf1fda3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:17.483418 32252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 934c68ab24ae453c8c373e811bc1e9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:17.483709 32100 tablet_service.cc:1467] Processing CreateTablet for tablet 934c68ab24ae453c8c373e811bc1e9eb (DEFAULT_TABLE table=test-table [id=166229d4910f4369a4293898daf1fda3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:17.483747 31946 tablet_service.cc:1467] Processing CreateTablet for tablet 934c68ab24ae453c8c373e811bc1e9eb (DEFAULT_TABLE table=test-table [id=166229d4910f4369a4293898daf1fda3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:17.485118 32100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 934c68ab24ae453c8c373e811bc1e9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:17.485107 31946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 934c68ab24ae453c8c373e811bc1e9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:17.495802 32345 tablet_bootstrap.cc:492] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: Bootstrap starting.
I20260811 03:16:17.499361 32345 tablet_bootstrap.cc:654] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:17.499953 32346 tablet_bootstrap.cc:492] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: Bootstrap starting.
I20260811 03:16:17.500877 32345 log.cc:824] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:17.501577 32347 tablet_bootstrap.cc:492] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: Bootstrap starting.
I20260811 03:16:17.504638 32346 tablet_bootstrap.cc:654] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:17.505467 32347 tablet_bootstrap.cc:654] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:17.506414 32346 log.cc:824] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:17.506510 32347 log.cc:824] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:17.509424 32347 tablet_bootstrap.cc:492] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: No bootstrap required, opened a new log
I20260811 03:16:17.509713 32347 ts_tablet_manager.cc:1397] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260811 03:16:17.510843 32345 tablet_bootstrap.cc:492] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: No bootstrap required, opened a new log
I20260811 03:16:17.510922 32346 tablet_bootstrap.cc:492] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: No bootstrap required, opened a new log
I20260811 03:16:17.511140 32345 ts_tablet_manager.cc:1397] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.012s
I20260811 03:16:17.511235 32346 ts_tablet_manager.cc:1397] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 03:16:17.519630 32345 raft_consensus.cc:348] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 } } peers { permanent_uuid: "488bda0764eb495fb7f26ada67276ecb" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42783 } } peers { permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 } }
I20260811 03:16:17.520059 32345 raft_consensus.cc:374] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:17.520187 32345 raft_consensus.cc:729] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 488bda0764eb495fb7f26ada67276ecb, State: Initialized, Role: FOLLOWER
I20260811 03:16:17.520499 32347 raft_consensus.cc:348] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 } } peers { permanent_uuid: "488bda0764eb495fb7f26ada67276ecb" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42783 } } peers { permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 } }
I20260811 03:16:17.520920 32347 raft_consensus.cc:374] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:17.521032 32347 raft_consensus.cc:729] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2944e5a53c040b08f60f646b744f8a6, State: Initialized, Role: FOLLOWER
I20260811 03:16:17.520902 32345 consensus_queue.cc:260] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 } } peers { permanent_uuid: "488bda0764eb495fb7f26ada67276ecb" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42783 } } peers { permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 } }
I20260811 03:16:17.521692 32347 consensus_queue.cc:260] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 } } peers { permanent_uuid: "488bda0764eb495fb7f26ada67276ecb" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42783 } } peers { permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 } }
I20260811 03:16:17.522184 32323 heartbeater.cc:503] Master 127.11.60.62:36197 was elected leader, sending a full tablet report...
I20260811 03:16:17.522291 32346 raft_consensus.cc:348] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 } } peers { permanent_uuid: "488bda0764eb495fb7f26ada67276ecb" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42783 } } peers { permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 } }
I20260811 03:16:17.522934 32346 raft_consensus.cc:374] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:17.522991 32345 ts_tablet_manager.cc:1428] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260811 03:16:17.523110 32346 raft_consensus.cc:729] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9fc0bb73884deebe943536ffc685f1, State: Initialized, Role: FOLLOWER
I20260811 03:16:17.523878 32346 consensus_queue.cc:260] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 } } peers { permanent_uuid: "488bda0764eb495fb7f26ada67276ecb" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42783 } } peers { permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 } }
I20260811 03:16:17.524742 32347 ts_tablet_manager.cc:1428] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260811 03:16:17.526854 32175 heartbeater.cc:503] Master 127.11.60.62:36197 was elected leader, sending a full tablet report...
I20260811 03:16:17.527408 32346 ts_tablet_manager.cc:1428] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260811 03:16:17.576807 32176 tablet.cc:2369] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:16:17.594799 32352 raft_consensus.cc:482] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:16:17.595127 32352 raft_consensus.cc:504] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 } } peers { permanent_uuid: "488bda0764eb495fb7f26ada67276ecb" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42783 } } peers { permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 } }
I20260811 03:16:17.597224 32352 leader_election.cc:288] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2944e5a53c040b08f60f646b744f8a6 (127.11.60.1:40093), ad9fc0bb73884deebe943536ffc685f1 (127.11.60.2:46231)
I20260811 03:16:17.604545 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.598098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:46303 (local address 127.11.60.1:40093)
0811 03:16:17.598322 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:17.598331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:17.598355 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:16:17.598540 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:17.598544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:17.598605 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:17.598685 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:17.599250 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.600351 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.602230 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.603529 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.603664 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.603780 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:17.604100 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 03:16:17.604288 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:16:17.604672 32326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.597542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:40093 (local address 127.11.60.3:46303)
0811 03:16:17.598167 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:17.598186 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:17.598340 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:17.598726 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:17.598730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:17.598856 (+ 126us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:17.599077 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.599088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.600483 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.600489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.602057 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.602077 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.603714 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.603720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.603908 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.603957 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 03:16:17.604057 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 03:16:17.604277 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490}
I20260811 03:16:17.605701 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.597644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:46231 (local address 127.11.60.3:48813)
0811 03:16:17.599119 (+ 1475us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:17.599134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:17.599282 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:17.599641 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:17.599646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:17.599671 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:17.600018 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.600028 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.601430 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.601436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.603084 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:17.603096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.604825 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.604831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:17.605026 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.605063 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 03:16:17.605203 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 03:16:17.605360 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1371,"thread_start_us":722,"threads_started":1}
I20260811 03:16:17.605703 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.597807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:48813 (local address 127.11.60.2:46231)
0811 03:16:17.598010 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:17.598017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:17.599237 (+ 1220us) server_negotiation.cc:407] Connection header received
0811 03:16:17.599422 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:17.599428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:17.599511 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:17.599601 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:17.600200 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.601270 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.603259 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.604456 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.604593 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.604687 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:17.605292 (+ 605us) server_negotiation.cc:299] Negotiation successful
0811 03:16:17.605459 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 03:16:17.606122 31967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "934c68ab24ae453c8c373e811bc1e9eb" candidate_uuid: "488bda0764eb495fb7f26ada67276ecb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2944e5a53c040b08f60f646b744f8a6" is_pre_election: true
I20260811 03:16:17.606773 31967 raft_consensus.cc:2454] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 488bda0764eb495fb7f26ada67276ecb in term 0.
I20260811 03:16:17.606958 32122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "934c68ab24ae453c8c373e811bc1e9eb" candidate_uuid: "488bda0764eb495fb7f26ada67276ecb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9fc0bb73884deebe943536ffc685f1" is_pre_election: true
I20260811 03:16:17.607489 32122 raft_consensus.cc:2454] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 488bda0764eb495fb7f26ada67276ecb in term 0.
I20260811 03:16:17.607573 32203 leader_election.cc:302] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 488bda0764eb495fb7f26ada67276ecb, d2944e5a53c040b08f60f646b744f8a6; no voters:
I20260811 03:16:17.608006 32352 raft_consensus.cc:2781] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:16:17.608186 32352 raft_consensus.cc:482] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:16:17.608307 32352 raft_consensus.cc:3037] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:17.610800 32352 raft_consensus.cc:504] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 } } peers { permanent_uuid: "488bda0764eb495fb7f26ada67276ecb" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42783 } } peers { permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 } }
I20260811 03:16:17.611450 32352 leader_election.cc:288] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [CANDIDATE]: Term 1 election: Requested vote from peers d2944e5a53c040b08f60f646b744f8a6 (127.11.60.1:40093), ad9fc0bb73884deebe943536ffc685f1 (127.11.60.2:46231)
I20260811 03:16:17.611853 31967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "934c68ab24ae453c8c373e811bc1e9eb" candidate_uuid: "488bda0764eb495fb7f26ada67276ecb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2944e5a53c040b08f60f646b744f8a6"
I20260811 03:16:17.612054 31967 raft_consensus.cc:3037] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:17.612272 32122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "934c68ab24ae453c8c373e811bc1e9eb" candidate_uuid: "488bda0764eb495fb7f26ada67276ecb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9fc0bb73884deebe943536ffc685f1"
I20260811 03:16:17.612613 32122 raft_consensus.cc:3037] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:17.615126 32122 raft_consensus.cc:2454] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 488bda0764eb495fb7f26ada67276ecb in term 1.
I20260811 03:16:17.615216 31967 raft_consensus.cc:2454] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 488bda0764eb495fb7f26ada67276ecb in term 1.
I20260811 03:16:17.615629 32201 leader_election.cc:302] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 488bda0764eb495fb7f26ada67276ecb, ad9fc0bb73884deebe943536ffc685f1; no voters:
I20260811 03:16:17.615962 32352 raft_consensus.cc:2781] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:17.616470 32352 raft_consensus.cc:686] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 1 LEADER]: Becoming Leader. State: Replica: 488bda0764eb495fb7f26ada67276ecb, State: Running, Role: LEADER
I20260811 03:16:17.617040 32352 consensus_queue.cc:237] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 } } peers { permanent_uuid: "488bda0764eb495fb7f26ada67276ecb" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42783 } } peers { permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 } }
I20260811 03:16:17.622045 31808 catalog_manager.cc:5495] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb reported cstate change: term changed from 0 to 1, leader changed from <none> to 488bda0764eb495fb7f26ada67276ecb (127.11.60.3). New cstate: current_term: 1 leader_uuid: "488bda0764eb495fb7f26ada67276ecb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "488bda0764eb495fb7f26ada67276ecb" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 42783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 } health_report { overall_health: UNKNOWN } } }
W20260811 03:16:17.628063 32324 tablet.cc:2369] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:16:17.667245 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.661795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.11.60.3:42783)
0811 03:16:17.661994 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:17.661998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:17.662017 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:16:17.662183 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:17.662187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:17.662264 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:17.662477 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:16:17.662859 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.663613 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.664962 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.665415 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.665512 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.665984 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:16:17.666630 (+ 646us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:16:17.666776 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:17.666913 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 03:16:17.666955 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 03:16:17.675498 32122 raft_consensus.cc:1261] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1 [term 1 FOLLOWER]: Refusing update from remote peer 488bda0764eb495fb7f26ada67276ecb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:17.675529 31967 raft_consensus.cc:1261] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6 [term 1 FOLLOWER]: Refusing update from remote peer 488bda0764eb495fb7f26ada67276ecb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:17.676450 32357 consensus_queue.cc:1035] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9fc0bb73884deebe943536ffc685f1" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 46231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:17.676769 32352 consensus_queue.cc:1035] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2944e5a53c040b08f60f646b744f8a6" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 40093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:17.687544 32361 mvcc.cc:205] Tried to move back new op lower bound from 7317168855748018176 to 7317168855524003840. Current Snapshot: MvccSnapshot[applied={T|T < 7317168855748018176}]
I20260811 03:16:17.693322 32362 mvcc.cc:205] Tried to move back new op lower bound from 7317168855748018176 to 7317168855524003840. Current Snapshot: MvccSnapshot[applied={T|T < 7317168855748018176}]
I20260811 03:16:17.700415 31808 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:17.700686 31808 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:17.704823 31808 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82b431d9c4a0488d9b7017d760c15fc1: Sending DeleteTablet for 3 replicas of tablet 934c68ab24ae453c8c373e811bc1e9eb
I20260811 03:16:17.706000 32252 tablet_service.cc:1514] Processing DeleteTablet for tablet 934c68ab24ae453c8c373e811bc1e9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:17 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:36848
I20260811 03:16:17.706004 31946 tablet_service.cc:1514] Processing DeleteTablet for tablet 934c68ab24ae453c8c373e811bc1e9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:17 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:48766
I20260811 03:16:17.707022 32100 tablet_service.cc:1514] Processing DeleteTablet for tablet 934c68ab24ae453c8c373e811bc1e9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:17 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:34976
I20260811 03:16:17.708009 32099 tablet_service.cc:1514] Processing DeleteTablet for tablet 934c68ab24ae453c8c373e811bc1e9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:17 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:34976
I20260811 03:16:17.712428 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 31876
I20260811 03:16:17.736289 32369 tablet_replica.cc:330] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: stopping tablet replica
I20260811 03:16:17.736768 32369 raft_consensus.cc:2229] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:16:17.737510 32369 raft_consensus.cc:2258] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:17.739650 32370 tablet_replica.cc:330] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: stopping tablet replica
I20260811 03:16:17.740152 32370 raft_consensus.cc:2229] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:17.740433 32369 ts_tablet_manager.cc:1905] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:17.740708 32370 raft_consensus.cc:2258] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:17.744104 32371 tablet_replica.cc:330] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: stopping tablet replica
I20260811 03:16:17.744505 32371 raft_consensus.cc:2229] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:17.745046 32371 raft_consensus.cc:2258] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:17.750025 32371 ts_tablet_manager.cc:1905] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:17.750679 32369 ts_tablet_manager.cc:1918] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:17.750878 32369 log.cc:1196] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/934c68ab24ae453c8c373e811bc1e9eb
I20260811 03:16:17.751448 32369 ts_tablet_manager.cc:1939] T 934c68ab24ae453c8c373e811bc1e9eb P 488bda0764eb495fb7f26ada67276ecb: Deleting consensus metadata
I20260811 03:16:17.751947 32370 ts_tablet_manager.cc:1905] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:17.753269 31792 catalog_manager.cc:4841] TS 488bda0764eb495fb7f26ada67276ecb (127.11.60.3:42783): tablet 934c68ab24ae453c8c373e811bc1e9eb (table test-table [id=166229d4910f4369a4293898daf1fda3]) successfully deleted
I20260811 03:16:17.756034 32371 ts_tablet_manager.cc:1918] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:17.756215 32371 log.cc:1196] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/934c68ab24ae453c8c373e811bc1e9eb
I20260811 03:16:17.756721 32371 ts_tablet_manager.cc:1939] T 934c68ab24ae453c8c373e811bc1e9eb P ad9fc0bb73884deebe943536ffc685f1: Deleting consensus metadata
I20260811 03:16:17.758291 31794 catalog_manager.cc:4841] TS ad9fc0bb73884deebe943536ffc685f1 (127.11.60.2:46231): tablet 934c68ab24ae453c8c373e811bc1e9eb (table test-table [id=166229d4910f4369a4293898daf1fda3]) successfully deleted
I20260811 03:16:17.759054 32370 ts_tablet_manager.cc:1918] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:17.759223 32370 log.cc:1196] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/934c68ab24ae453c8c373e811bc1e9eb
I20260811 03:16:17.759788 32370 ts_tablet_manager.cc:1939] T 934c68ab24ae453c8c373e811bc1e9eb P d2944e5a53c040b08f60f646b744f8a6: Deleting consensus metadata
Aug 11 03:16:17 dist-test-slave-tm1g krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
W20260811 03:16:17.761219 31794 catalog_manager.cc:4808] TS ad9fc0bb73884deebe943536ffc685f1 (127.11.60.2:46231): delete failed for tablet 934c68ab24ae453c8c373e811bc1e9eb because the tablet was not found. No further retry: Not found: Tablet not found: 934c68ab24ae453c8c373e811bc1e9eb
I20260811 03:16:17.761590 31795 catalog_manager.cc:4841] TS d2944e5a53c040b08f60f646b744f8a6 (127.11.60.1:40093): tablet 934c68ab24ae453c8c373e811bc1e9eb (table test-table [id=166229d4910f4369a4293898daf1fda3]) successfully deleted
I20260811 03:16:17.768725 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.713692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.11.60.1:40093)
0811 03:16:17.713873 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:17.713879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:17.733260 (+ 19381us) server_negotiation.cc:407] Connection header received
0811 03:16:17.733875 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:17.733884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:17.734056 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:17.734153 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:17.741017 (+ 6864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.742181 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.746476 (+ 4295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.747226 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.747334 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.762624 (+ 15290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:17.762636 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:17.762650 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:17.762686 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:17.765513 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:17.766523 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:17.766530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:17.766532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:17.766665 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:17.767099 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:17.767105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:17.767112 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:17.767468 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:17.767790 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:17.768119 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 03:16:17.768362 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 03:16:17.769362 32015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:17.898393 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 31876
I20260811 03:16:17.920619 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 32029
Aug 11 03:16:17 dist-test-slave-tm1g krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:16:17.938419 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:17.921535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34984 (local address 127.11.60.2:46231)
0811 03:16:17.921770 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:17.921777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:17.921801 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:16:17.921876 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:17.921883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:17.921993 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:17.922105 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:17.923477 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.924733 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.926045 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:17.926854 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:17.926987 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:17.931286 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:17.931307 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:17.931329 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:17.931380 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:17.935299 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:17.936049 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:17.936060 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:17.936065 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:17.936249 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:17.936682 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:17.936687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:17.936696 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:17.937206 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:17.937495 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:17.937779 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 03:16:17.938016 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 03:16:17.939281 32169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:18.030579 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 32029
I20260811 03:16:18.052238 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 32183
Aug 11 03:16:18 dist-test-slave-tm1g krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:16:18.069712 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:18.053101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.11.60.3:42783)
0811 03:16:18.053286 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:18.053294 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:18.053314 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:16:18.053559 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:18.053564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:18.053631 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:18.053750 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:18.055175 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:18.056038 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:18.057344 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:18.057803 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:18.058013 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:18.062954 (+ 4941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:18.062972 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:18.062993 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:18.063025 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:18.066488 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:18.067378 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:18.067391 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:18.067393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:18.067584 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:18.068065 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:18.068069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:18.068071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:18.068459 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:18.068718 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:18.069076 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 03:16:18.069246 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:16:18.070709 32318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:18.198205 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 32183
I20260811 03:16:18.221484 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 31772
I20260811 03:16:18.222723 31842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:18.338786 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 31772
2026-08-11T03:16:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4719 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T03:16:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:18Z Disabled control of system clock
I20260811 03:16:18.434504 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:41423
--webserver_interface=127.11.60.62
--webserver_port=0
--builtin_ntp_servers=127.11.60.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:41423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:16:18.785238 32380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:18.785581 32380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:18.785758 32380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:18.793296 32380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:18.793411 32380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:18.793488 32380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:18.793606 32380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:18.793694 32380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:18.803224 32380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:41423
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:41423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:18.804663 32380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:18.806631 32380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:18.819644 32387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:18.819638 32389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:18.819697 32386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:18.820962 32380 server_base.cc:1034] running on GCE node
I20260811 03:16:18.821870 32380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:18.823612 32380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:18.824903 32380 hybrid_clock.cc:648] HybridClock initialized: now 1786418178824878 us; error 124 us; skew 500 ppm
I20260811 03:16:18.825361 32380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:18.833211 32380 webserver.cc:460] Webserver started at http://127.11.60.62:33279/ using document root <none> and password file <none>
I20260811 03:16:18.834041 32380 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:18.834141 32380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:18.834575 32380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:18.837679 32380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "1248d6a014144d59bf3b6d51aeffe775"
format_stamp: "Formatted at 2026-08-11 03:16:18 on dist-test-slave-tm1g"
server_key: "4e6da1e267e7460adcb262d4e3b59790"
server_key_iv: "942780f62ddee07324d6a46fd31ff601"
server_key_version: "encryptionkey@0"
I20260811 03:16:18.838655 32380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "1248d6a014144d59bf3b6d51aeffe775"
format_stamp: "Formatted at 2026-08-11 03:16:18 on dist-test-slave-tm1g"
server_key: "4e6da1e267e7460adcb262d4e3b59790"
server_key_iv: "942780f62ddee07324d6a46fd31ff601"
server_key_version: "encryptionkey@0"
I20260811 03:16:18.844022 32380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:16:18.847532 32397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:18.848872 32380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:16:18.849071 32380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "1248d6a014144d59bf3b6d51aeffe775"
format_stamp: "Formatted at 2026-08-11 03:16:18 on dist-test-slave-tm1g"
server_key: "4e6da1e267e7460adcb262d4e3b59790"
server_key_iv: "942780f62ddee07324d6a46fd31ff601"
server_key_version: "encryptionkey@0"
I20260811 03:16:18.849296 32380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:18.873315 32380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:18.874243 32380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:18.874588 32380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:18.893921 32380 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:41423
I20260811 03:16:18.893971 32456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:41423 every 8 connection(s)
I20260811 03:16:18.895375 32380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:18.899250 32457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:18.902927 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 32380
I20260811 03:16:18.903112 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:18.903703 11504 external_mini_cluster.cc:1428] Setting key 64478bc84dcd6c20f69848fec99fbdba
I20260811 03:16:18.912262 32457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775: Bootstrap starting.
I20260811 03:16:18.916039 32457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:18.917289 32457 log.cc:824] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:18.918325 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:18.905724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53250 (local address 127.11.60.62:41423)
0811 03:16:18.906716 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:18.906733 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:18.906775 (+ 42us) server_negotiation.cc:407] Connection header received
0811 03:16:18.908350 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:18.908370 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:18.908860 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:18.909189 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:18.909831 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:18.911489 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:18.913431 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:18.914171 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:18.914437 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:18.915659 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:18.915667 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:18.915680 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:18.916274 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:18.916375 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:18.916430 (+ 55us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:18.916531 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:18.916754 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 03:16:18.917096 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":374,"threads_started":1}
I20260811 03:16:18.920578 32457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775: No bootstrap required, opened a new log
I20260811 03:16:18.928107 32457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248d6a014144d59bf3b6d51aeffe775" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41423 } }
I20260811 03:16:18.928483 32457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:18.928591 32457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1248d6a014144d59bf3b6d51aeffe775, State: Initialized, Role: FOLLOWER
I20260811 03:16:18.929263 32457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248d6a014144d59bf3b6d51aeffe775" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41423 } }
I20260811 03:16:18.929563 32457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:18.929756 32457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:18.929949 32457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:18.932118 32457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248d6a014144d59bf3b6d51aeffe775" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41423 } }
I20260811 03:16:18.932629 32457 leader_election.cc:302] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1248d6a014144d59bf3b6d51aeffe775; no voters:
I20260811 03:16:18.933235 32457 leader_election.cc:288] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:18.933487 32462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:18.934278 32462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [term 1 LEADER]: Becoming Leader. State: Replica: 1248d6a014144d59bf3b6d51aeffe775, State: Running, Role: LEADER
I20260811 03:16:18.934866 32462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248d6a014144d59bf3b6d51aeffe775" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41423 } }
I20260811 03:16:18.935271 32457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:18.937902 32464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1248d6a014144d59bf3b6d51aeffe775. Latest consensus state: current_term: 1 leader_uuid: "1248d6a014144d59bf3b6d51aeffe775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248d6a014144d59bf3b6d51aeffe775" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41423 } } }
I20260811 03:16:18.938623 32464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:18.938256 32463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1248d6a014144d59bf3b6d51aeffe775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248d6a014144d59bf3b6d51aeffe775" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 41423 } } }
I20260811 03:16:18.938746 32463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:18.942698 32470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:18.947728 32470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:18.956882 32470 catalog_manager.cc:1347] Generated new cluster ID: 2ca03ab20a7f403b8f142dfbb1d7982d
I20260811 03:16:18.957001 32470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:19.008500 32470 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:19.010476 32470 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:19.021277 32470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775: Generated new TSK 0
I20260811 03:16:19.022368 32470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:19.033895 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41423
--builtin_ntp_servers=127.11.60.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:16:19.383288 32483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:19.383584 32483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:19.383742 32483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:19.391050 32483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:19.391148 32483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:19.391320 32483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:16:19.400312 32483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:19.401676 32483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:19.403486 32483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:19.418483 32492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:19.418429 32489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:19.418488 32490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:19.424117 32483 server_base.cc:1034] running on GCE node
I20260811 03:16:19.424794 32483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:19.425812 32483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:19.427062 32483 hybrid_clock.cc:648] HybridClock initialized: now 1786418179426932 us; error 118 us; skew 500 ppm
I20260811 03:16:19.427397 32483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:19.429659 32483 webserver.cc:460] Webserver started at http://127.11.60.1:37985/ using document root <none> and password file <none>
I20260811 03:16:19.430487 32483 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:19.430624 32483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:19.430950 32483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:19.433902 32483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "de8d9ecfdd904ce38a4a165508f8db54"
format_stamp: "Formatted at 2026-08-11 03:16:19 on dist-test-slave-tm1g"
server_key: "e34fd84fc3b056cf81a70a28d975dd93"
server_key_iv: "f73b9dadcc1fb33652039ed3ea454c35"
server_key_version: "encryptionkey@0"
I20260811 03:16:19.434767 32483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "de8d9ecfdd904ce38a4a165508f8db54"
format_stamp: "Formatted at 2026-08-11 03:16:19 on dist-test-slave-tm1g"
server_key: "e34fd84fc3b056cf81a70a28d975dd93"
server_key_iv: "f73b9dadcc1fb33652039ed3ea454c35"
server_key_version: "encryptionkey@0"
I20260811 03:16:19.440479 32483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:16:19.444200 32499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:19.445578 32483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:16:19.445760 32483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "de8d9ecfdd904ce38a4a165508f8db54"
format_stamp: "Formatted at 2026-08-11 03:16:19 on dist-test-slave-tm1g"
server_key: "e34fd84fc3b056cf81a70a28d975dd93"
server_key_iv: "f73b9dadcc1fb33652039ed3ea454c35"
server_key_version: "encryptionkey@0"
I20260811 03:16:19.445977 32483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:19.459028 32483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:19.459811 32483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:19.460095 32483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:19.461124 32483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:19.462653 32483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:19.462781 32483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:19.462920 32483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:19.463011 32483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:19.498060 32483 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:37223
I20260811 03:16:19.498109 32629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:37223 every 8 connection(s)
I20260811 03:16:19.499459 32483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:16:19.508203 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 32483
I20260811 03:16:19.508412 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:16:19.508869 11504 external_mini_cluster.cc:1428] Setting key c965f265e99a7ce5ab8d2002f35ff7b9
I20260811 03:16:19.511915 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41423
--builtin_ntp_servers=127.11.60.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:16:19.512795 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:19.502220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:42609 (local address 127.11.60.62:41423)
0811 03:16:19.502745 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:19.502749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:19.503905 (+ 1156us) server_negotiation.cc:407] Connection header received
0811 03:16:19.505722 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:19.505726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:19.505793 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:19.505894 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:19.507141 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:19.507986 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:19.509781 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:19.510251 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:19.510457 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:19.511896 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:19.511903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:19.511910 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:19.511934 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:19.511991 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:19.512011 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:19.512122 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:19.512450 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 03:16:19.512533 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":281,"threads_started":1}
I20260811 03:16:19.514351 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:19.502816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:41423 (local address 127.11.60.1:42609)
0811 03:16:19.503733 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:19.503777 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:19.505467 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:19.506149 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:19.506157 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:19.506245 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:19.506983 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:19.507001 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:19.508161 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:19.508165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:19.509637 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:19.509650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:19.510545 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:19.510553 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:19.510822 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:19.511560 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:16:19.511563 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:19.511604 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:16:19.511682 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:19.512185 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:19.512200 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:19.512211 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 03:16:19.512415 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 03:16:19.512847 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":302,"threads_started":1}
I20260811 03:16:19.516088 32630 heartbeater.cc:347] Connected to a master server at 127.11.60.62:41423
I20260811 03:16:19.516424 32630 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:19.517191 32630 heartbeater.cc:511] Master 127.11.60.62:41423 requested a full tablet report, sending...
I20260811 03:16:19.519248 32415 ts_manager.cc:194] Registered new tserver with Master: de8d9ecfdd904ce38a4a165508f8db54 (127.11.60.1:37223)
I20260811 03:16:19.520844 32415 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.1:42609
W20260811 03:16:19.856753 32634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:19.857033 32634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:19.857121 32634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:19.864353 32634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:19.864451 32634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:19.864557 32634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:16:19.873404 32634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:19.874698 32634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:19.876422 32634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:19.889819 32639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:19.889843 32640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:19.889899 32642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:19.891149 32634 server_base.cc:1034] running on GCE node
I20260811 03:16:19.891795 32634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:19.892845 32634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:19.894140 32634 hybrid_clock.cc:648] HybridClock initialized: now 1786418179894117 us; error 70 us; skew 500 ppm
I20260811 03:16:19.894531 32634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:19.896917 32634 webserver.cc:460] Webserver started at http://127.11.60.2:43705/ using document root <none> and password file <none>
I20260811 03:16:19.897624 32634 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:19.897709 32634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:19.898061 32634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:19.900935 32634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "58c95847ec024adab215b07120dbfe56"
format_stamp: "Formatted at 2026-08-11 03:16:19 on dist-test-slave-tm1g"
server_key: "f388a2ca70ca56d592d9d5b01d9d98e2"
server_key_iv: "5e883864ffd539f06eb0c466dfc184f2"
server_key_version: "encryptionkey@0"
I20260811 03:16:19.901715 32634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "58c95847ec024adab215b07120dbfe56"
format_stamp: "Formatted at 2026-08-11 03:16:19 on dist-test-slave-tm1g"
server_key: "f388a2ca70ca56d592d9d5b01d9d98e2"
server_key_iv: "5e883864ffd539f06eb0c466dfc184f2"
server_key_version: "encryptionkey@0"
I20260811 03:16:19.906970 32634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:16:19.910574 32649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:19.911942 32634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:16:19.912112 32634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "58c95847ec024adab215b07120dbfe56"
format_stamp: "Formatted at 2026-08-11 03:16:19 on dist-test-slave-tm1g"
server_key: "f388a2ca70ca56d592d9d5b01d9d98e2"
server_key_iv: "5e883864ffd539f06eb0c466dfc184f2"
server_key_version: "encryptionkey@0"
I20260811 03:16:19.912333 32634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:19.929041 32634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:19.929795 32634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:19.930032 32634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:19.931100 32634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:19.932499 32634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:19.932588 32634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:19.932720 32634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:19.932811 32634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:19.968299 32634 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:42921
I20260811 03:16:19.968355 308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:42921 every 8 connection(s)
I20260811 03:16:19.969689 32634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:16:19.973785 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 32634
I20260811 03:16:19.973989 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:16:19.974633 11504 external_mini_cluster.cc:1428] Setting key d9a288e05ae07cffb8f3ff9a37b7b2c8
I20260811 03:16:19.977849 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41423
--builtin_ntp_servers=127.11.60.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:16:19.983691 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:19.972373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:49785 (local address 127.11.60.62:41423)
0811 03:16:19.972593 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:19.972599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:19.974194 (+ 1595us) server_negotiation.cc:407] Connection header received
0811 03:16:19.976447 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:19.976451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:19.976515 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:19.976653 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:19.977761 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:19.978965 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:19.980524 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:19.981246 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:19.981382 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:19.982784 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:19.982793 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:19.982800 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:19.982825 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:19.982880 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:19.982899 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:19.982994 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:19.983192 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 03:16:19.983303 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:16:19.984514 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:19.972913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:41423 (local address 127.11.60.2:49785)
0811 03:16:19.974022 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:19.974070 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:19.976134 (+ 2064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:19.976914 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:19.976925 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:19.977033 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:19.977591 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:19.977607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:19.979106 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:19.979111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:19.980361 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:19.980373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:19.981483 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:19.981489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:19.981686 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:19.982477 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:16:19.982480 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:19.982525 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:16:19.982597 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:19.982997 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:19.983005 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:19.983010 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 03:16:19.983146 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 03:16:19.983453 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":300,"threads_started":1}
I20260811 03:16:19.986060 309 heartbeater.cc:347] Connected to a master server at 127.11.60.62:41423
I20260811 03:16:19.986423 309 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:19.987340 309 heartbeater.cc:511] Master 127.11.60.62:41423 requested a full tablet report, sending...
I20260811 03:16:19.988955 32415 ts_manager.cc:194] Registered new tserver with Master: 58c95847ec024adab215b07120dbfe56 (127.11.60.2:42921)
I20260811 03:16:19.990039 32415 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.2:49785
W20260811 03:16:20.324144 312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:20.324427 312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:20.324580 312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:20.331847 312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:20.331939 312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:20.332108 312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:16:20.342492 312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:41423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:20.343851 312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:20.345661 312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:20.359885 319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:20.359819 318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:20.360949 321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:20.363955 312 server_base.cc:1034] running on GCE node
I20260811 03:16:20.364637 312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:20.365739 312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:20.367012 312 hybrid_clock.cc:648] HybridClock initialized: now 1786418180366965 us; error 59 us; skew 500 ppm
I20260811 03:16:20.367376 312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:20.369792 312 webserver.cc:460] Webserver started at http://127.11.60.3:43113/ using document root <none> and password file <none>
I20260811 03:16:20.370627 312 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:20.370764 312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:20.371094 312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:20.373965 312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "b0fa7723416a49448f20e68ed84a28fd"
format_stamp: "Formatted at 2026-08-11 03:16:20 on dist-test-slave-tm1g"
server_key: "d8372460e475f2ada2bd9600d45b4552"
server_key_iv: "7736a4599467a71392e9c4381f6b5df4"
server_key_version: "encryptionkey@0"
I20260811 03:16:20.374771 312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "b0fa7723416a49448f20e68ed84a28fd"
format_stamp: "Formatted at 2026-08-11 03:16:20 on dist-test-slave-tm1g"
server_key: "d8372460e475f2ada2bd9600d45b4552"
server_key_iv: "7736a4599467a71392e9c4381f6b5df4"
server_key_version: "encryptionkey@0"
I20260811 03:16:20.379842 312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:16:20.383294 327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:20.384555 312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:16:20.384742 312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "b0fa7723416a49448f20e68ed84a28fd"
format_stamp: "Formatted at 2026-08-11 03:16:20 on dist-test-slave-tm1g"
server_key: "d8372460e475f2ada2bd9600d45b4552"
server_key_iv: "7736a4599467a71392e9c4381f6b5df4"
server_key_version: "encryptionkey@0"
I20260811 03:16:20.384963 312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:20.396386 312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:20.397181 312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:20.397473 312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:20.398497 312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:20.399919 312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:20.400041 312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:20.400177 312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:20.400264 312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:20.436434 312 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:37841
I20260811 03:16:20.436527 452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:37841 every 8 connection(s)
I20260811 03:16:20.437858 312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:16:20.441679 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 312
I20260811 03:16:20.441875 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:16:20.442464 11504 external_mini_cluster.cc:1428] Setting key f21d0e4ace5fd8878897bc2afe716f78
I20260811 03:16:20.451217 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.440551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:34401 (local address 127.11.60.62:41423)
0811 03:16:20.440747 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.440751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.442512 (+ 1761us) server_negotiation.cc:407] Connection header received
0811 03:16:20.444529 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.444534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.444620 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.444742 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:20.445808 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.446680 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.448244 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.448684 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.448825 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.450277 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:20.450284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:20.450288 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:20.450341 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:20.450383 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:20.450406 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:20.450500 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.450735 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.450837 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:16:20.452195 455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.441125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:41423 (local address 127.11.60.3:34401)
0811 03:16:20.442359 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:20.442405 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:20.444266 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:20.444982 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:20.444991 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:20.445076 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:20.445642 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.445658 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.446833 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.446844 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.448096 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.448108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.448917 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.448924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.449173 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.449949 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:16:20.449952 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:20.449989 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:16:20.450064 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:20.450517 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:20.450531 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:20.450544 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 03:16:20.450718 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 03:16:20.451118 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":432,"threads_started":1}
I20260811 03:16:20.453564 453 heartbeater.cc:347] Connected to a master server at 127.11.60.62:41423
I20260811 03:16:20.453914 453 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:20.454756 453 heartbeater.cc:511] Master 127.11.60.62:41423 requested a full tablet report, sending...
I20260811 03:16:20.456437 32415 ts_manager.cc:194] Registered new tserver with Master: b0fa7723416a49448f20e68ed84a28fd (127.11.60.3:37841)
I20260811 03:16:20.457499 32415 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.3:34401
I20260811 03:16:20.458158 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:16:20.468516 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.462684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.11.60.62:41423)
0811 03:16:20.462910 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.462915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.463474 (+ 559us) server_negotiation.cc:407] Connection header received
0811 03:16:20.463561 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.463564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.463634 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.463719 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:20.464167 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.464859 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.466398 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.466803 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.466902 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.467916 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:20.467921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:20.467924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:20.467940 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:20.467984 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:20.467995 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:20.468102 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.468261 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.468304 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 03:16:20.472285 32415 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:16:20.479245 32415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:16:20.520563 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.491258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:37223 (local address 127.0.0.1:34310)
0811 03:16:20.510511 (+ 19253us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:20.510537 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:20.510701 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:20.511953 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:20.511977 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:20.512029 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:20.512604 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.512621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.515566 (+ 2945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.515574 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.517706 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.517718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.519574 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.519581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.519816 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.519868 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 03:16:20.520011 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 03:16:20.520190 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19025,"thread_start_us":9167,"threads_started":1}
I20260811 03:16:20.520634 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.491931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.11.60.1:37223)
0811 03:16:20.507002 (+ 15071us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.507010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.510868 (+ 3858us) server_negotiation.cc:407] Connection header received
0811 03:16:20.511025 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.511037 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.511357 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.511590 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:20.512872 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.515319 (+ 2447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.517900 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.519345 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.519511 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.519805 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.520074 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.520255 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14874,"thread_start_us":15045,"threads_started":1}
I20260811 03:16:20.523252 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.501459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:42921 (local address 127.0.0.1:41650)
0811 03:16:20.514609 (+ 13150us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:20.514631 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:20.514773 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:20.515804 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:20.515809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:20.515838 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:20.516296 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.516357 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.518359 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.518364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.520717 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.520734 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.522448 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.522456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.522645 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.522676 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 03:16:20.522761 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 03:16:20.522902 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13014,"mutex_wait_us":82,"thread_start_us":12744,"threads_started":1}
I20260811 03:16:20.523856 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.491254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39730 (local address 127.11.60.3:37841)
0811 03:16:20.514992 (+ 23738us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.515001 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.515032 (+ 31us) server_negotiation.cc:407] Connection header received
0811 03:16:20.515117 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.515123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.515299 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.515510 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:20.516792 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.518345 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.520747 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.522113 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.522252 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.522627 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.523350 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.523520 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23559,"thread_start_us":23405,"threads_started":1}
I20260811 03:16:20.523866 32630 heartbeater.cc:503] Master 127.11.60.62:41423 was elected leader, sending a full tablet report...
I20260811 03:16:20.524325 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.501541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41650 (local address 127.11.60.2:42921)
0811 03:16:20.513928 (+ 12387us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.513941 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.514909 (+ 968us) server_negotiation.cc:407] Connection header received
0811 03:16:20.515022 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.515030 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.515336 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.515510 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:20.516563 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.518189 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.521001 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.522180 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.523478 (+ 1298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.523735 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.523806 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.523981 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12165,"thread_start_us":12398,"threads_started":1}
I20260811 03:16:20.524400 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.491135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:37841 (local address 127.0.0.1:39730)
0811 03:16:20.500807 (+ 9672us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:20.500847 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:20.501036 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:20.515812 (+ 14776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:20.515816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:20.515856 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:20.516514 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.516528 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.518512 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.518517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.520477 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.520492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.522719 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.522730 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.522931 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.522963 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 03:16:20.523054 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 03:16:20.523192 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9428,"mutex_wait_us":55,"spinlock_wait_cycles":7040,"thread_start_us":18951,"threads_started":1}
I20260811 03:16:20.525496 32551 tablet_service.cc:1467] Processing CreateTablet for tablet 49badcb4a2db47e0a04103a74a3788a3 (DEFAULT_TABLE table=test-table [id=38cdffac9e6847d19f7250de37f9585b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:20.526926 32551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49badcb4a2db47e0a04103a74a3788a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:20.527848 32704 tablet_service.cc:1467] Processing CreateTablet for tablet 49badcb4a2db47e0a04103a74a3788a3 (DEFAULT_TABLE table=test-table [id=38cdffac9e6847d19f7250de37f9585b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:20.529254 32704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49badcb4a2db47e0a04103a74a3788a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:20.530170 379 tablet_service.cc:1467] Processing CreateTablet for tablet 49badcb4a2db47e0a04103a74a3788a3 (DEFAULT_TABLE table=test-table [id=38cdffac9e6847d19f7250de37f9585b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:20.531644 379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49badcb4a2db47e0a04103a74a3788a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:20.540956 473 tablet_bootstrap.cc:492] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: Bootstrap starting.
I20260811 03:16:20.544965 475 tablet_bootstrap.cc:492] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: Bootstrap starting.
I20260811 03:16:20.546553 474 tablet_bootstrap.cc:492] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: Bootstrap starting.
I20260811 03:16:20.547152 473 tablet_bootstrap.cc:654] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:20.548190 473 log.cc:824] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:20.548588 475 tablet_bootstrap.cc:654] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:20.549883 475 log.cc:824] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:20.549975 474 tablet_bootstrap.cc:654] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:20.551047 474 log.cc:824] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:20.551476 473 tablet_bootstrap.cc:492] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: No bootstrap required, opened a new log
I20260811 03:16:20.551864 473 ts_tablet_manager.cc:1397] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 03:16:20.553119 475 tablet_bootstrap.cc:492] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: No bootstrap required, opened a new log
I20260811 03:16:20.553469 475 ts_tablet_manager.cc:1397] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260811 03:16:20.554412 474 tablet_bootstrap.cc:492] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: No bootstrap required, opened a new log
I20260811 03:16:20.554658 474 ts_tablet_manager.cc:1397] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 03:16:20.560485 473 raft_consensus.cc:348] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fa7723416a49448f20e68ed84a28fd" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37841 } } peers { permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 } } peers { permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 } }
I20260811 03:16:20.561064 473 raft_consensus.cc:374] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:20.561200 473 raft_consensus.cc:729] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de8d9ecfdd904ce38a4a165508f8db54, State: Initialized, Role: FOLLOWER
I20260811 03:16:20.561920 473 consensus_queue.cc:260] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fa7723416a49448f20e68ed84a28fd" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37841 } } peers { permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 } } peers { permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 } }
I20260811 03:16:20.563787 473 ts_tablet_manager.cc:1428] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: Time spent starting tablet: real 0.012s user 0.009s sys 0.001s
I20260811 03:16:20.564743 475 raft_consensus.cc:348] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fa7723416a49448f20e68ed84a28fd" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37841 } } peers { permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 } } peers { permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 } }
I20260811 03:16:20.565199 475 raft_consensus.cc:374] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:20.565332 475 raft_consensus.cc:729] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c95847ec024adab215b07120dbfe56, State: Initialized, Role: FOLLOWER
I20260811 03:16:20.565285 474 raft_consensus.cc:348] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fa7723416a49448f20e68ed84a28fd" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37841 } } peers { permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 } } peers { permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 } }
I20260811 03:16:20.565810 474 raft_consensus.cc:374] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:20.565922 474 raft_consensus.cc:729] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fa7723416a49448f20e68ed84a28fd, State: Initialized, Role: FOLLOWER
I20260811 03:16:20.565960 475 consensus_queue.cc:260] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fa7723416a49448f20e68ed84a28fd" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37841 } } peers { permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 } } peers { permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 } }
I20260811 03:16:20.566534 474 consensus_queue.cc:260] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fa7723416a49448f20e68ed84a28fd" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37841 } } peers { permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 } } peers { permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 } }
I20260811 03:16:20.567214 309 heartbeater.cc:503] Master 127.11.60.62:41423 was elected leader, sending a full tablet report...
I20260811 03:16:20.567922 475 ts_tablet_manager.cc:1428] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 03:16:20.569082 453 heartbeater.cc:503] Master 127.11.60.62:41423 was elected leader, sending a full tablet report...
I20260811 03:16:20.571024 474 ts_tablet_manager.cc:1428] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260811 03:16:20.599218 482 raft_consensus.cc:482] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:16:20.599524 482 raft_consensus.cc:504] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fa7723416a49448f20e68ed84a28fd" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37841 } } peers { permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 } } peers { permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 } }
I20260811 03:16:20.601297 482 leader_election.cc:288] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de8d9ecfdd904ce38a4a165508f8db54 (127.11.60.1:37223), 58c95847ec024adab215b07120dbfe56 (127.11.60.2:42921)
I20260811 03:16:20.608230 455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.601688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:37223 (local address 127.11.60.3:37023)
0811 03:16:20.601931 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:20.601952 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:20.602140 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:20.602682 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:20.602690 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:20.602723 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:20.603010 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.603023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.604340 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.604345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.606272 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.606287 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.607637 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.607641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.607780 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.607805 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 03:16:20.607884 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 03:16:20.607994 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":24}
I20260811 03:16:20.608332 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.601720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:37023 (local address 127.11.60.1:37223)
0811 03:16:20.601941 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.601948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.602047 (+ 99us) server_negotiation.cc:407] Connection header received
0811 03:16:20.602373 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.602382 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.602487 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.602618 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:20.603215 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.604205 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.606476 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.607457 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.607599 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.607699 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.607943 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.608080 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:16:20.609246 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.602369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:54297 (local address 127.11.60.2:42921)
0811 03:16:20.602592 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.602599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.603214 (+ 615us) server_negotiation.cc:407] Connection header received
0811 03:16:20.603363 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.603368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.603503 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.603602 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:20.604071 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.604860 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.607093 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.608270 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.608475 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.608568 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.608971 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.609067 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 03:16:20.609417 483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.602140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:42921 (local address 127.11.60.3:54297)
0811 03:16:20.603097 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:20.603116 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:20.603227 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:20.603622 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:20.603625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:20.603643 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:20.603924 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.603931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.605032 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.605038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.606904 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:20.606914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.608660 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.608665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:20.608786 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.608828 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 03:16:20.608936 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 03:16:20.609072 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":28,"thread_start_us":654,"threads_started":1}
I20260811 03:16:20.609429 32575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49badcb4a2db47e0a04103a74a3788a3" candidate_uuid: "b0fa7723416a49448f20e68ed84a28fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de8d9ecfdd904ce38a4a165508f8db54" is_pre_election: true
I20260811 03:16:20.610071 32575 raft_consensus.cc:2454] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0fa7723416a49448f20e68ed84a28fd in term 0.
I20260811 03:16:20.610855 329 leader_election.cc:302] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0fa7723416a49448f20e68ed84a28fd, de8d9ecfdd904ce38a4a165508f8db54; no voters:
I20260811 03:16:20.610685 32728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49badcb4a2db47e0a04103a74a3788a3" candidate_uuid: "b0fa7723416a49448f20e68ed84a28fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c95847ec024adab215b07120dbfe56" is_pre_election: true
I20260811 03:16:20.611230 482 raft_consensus.cc:2781] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:16:20.611284 32728 raft_consensus.cc:2454] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0fa7723416a49448f20e68ed84a28fd in term 0.
I20260811 03:16:20.611426 482 raft_consensus.cc:482] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:16:20.611548 482 raft_consensus.cc:3037] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:20.614110 482 raft_consensus.cc:504] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fa7723416a49448f20e68ed84a28fd" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37841 } } peers { permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 } } peers { permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 } }
I20260811 03:16:20.614810 482 leader_election.cc:288] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [CANDIDATE]: Term 1 election: Requested vote from peers de8d9ecfdd904ce38a4a165508f8db54 (127.11.60.1:37223), 58c95847ec024adab215b07120dbfe56 (127.11.60.2:42921)
I20260811 03:16:20.615314 32575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49badcb4a2db47e0a04103a74a3788a3" candidate_uuid: "b0fa7723416a49448f20e68ed84a28fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de8d9ecfdd904ce38a4a165508f8db54"
I20260811 03:16:20.615346 32728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49badcb4a2db47e0a04103a74a3788a3" candidate_uuid: "b0fa7723416a49448f20e68ed84a28fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c95847ec024adab215b07120dbfe56"
I20260811 03:16:20.615514 32575 raft_consensus.cc:3037] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:20.615645 32728 raft_consensus.cc:3037] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:20.618121 32575 raft_consensus.cc:2454] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fa7723416a49448f20e68ed84a28fd in term 1.
I20260811 03:16:20.618589 329 leader_election.cc:302] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0fa7723416a49448f20e68ed84a28fd, de8d9ecfdd904ce38a4a165508f8db54; no voters:
I20260811 03:16:20.618649 32728 raft_consensus.cc:2454] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fa7723416a49448f20e68ed84a28fd in term 1.
I20260811 03:16:20.618912 482 raft_consensus.cc:2781] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:20.619491 482 raft_consensus.cc:686] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 1 LEADER]: Becoming Leader. State: Replica: b0fa7723416a49448f20e68ed84a28fd, State: Running, Role: LEADER
I20260811 03:16:20.620108 482 consensus_queue.cc:237] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fa7723416a49448f20e68ed84a28fd" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37841 } } peers { permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 } } peers { permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 } }
I20260811 03:16:20.624730 32414 catalog_manager.cc:5495] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd reported cstate change: term changed from 0 to 1, leader changed from <none> to b0fa7723416a49448f20e68ed84a28fd (127.11.60.3). New cstate: current_term: 1 leader_uuid: "b0fa7723416a49448f20e68ed84a28fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fa7723416a49448f20e68ed84a28fd" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 37841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 } health_report { overall_health: UNKNOWN } } }
I20260811 03:16:20.663702 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.657080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39740 (local address 127.11.60.3:37841)
0811 03:16:20.657475 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.657484 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.657572 (+ 88us) server_negotiation.cc:407] Connection header received
0811 03:16:20.657829 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.657834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.657934 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.658105 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:16:20.658639 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.659448 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.661049 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.661544 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.661691 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.662450 (+ 759us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:16:20.663043 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:16:20.663258 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.663410 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.663458 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260811 03:16:20.672149 32728 raft_consensus.cc:1261] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56 [term 1 FOLLOWER]: Refusing update from remote peer b0fa7723416a49448f20e68ed84a28fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:20.672221 32575 raft_consensus.cc:1261] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54 [term 1 FOLLOWER]: Refusing update from remote peer b0fa7723416a49448f20e68ed84a28fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:20.673101 484 consensus_queue.cc:1035] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c95847ec024adab215b07120dbfe56" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:20.673521 482 consensus_queue.cc:1035] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "de8d9ecfdd904ce38a4a165508f8db54" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:20.684942 488 mvcc.cc:205] Tried to move back new op lower bound from 7317168868022640640 to 7317168867824267264. Current Snapshot: MvccSnapshot[applied={T|T < 7317168868022640640}]
I20260811 03:16:20.687608 490 mvcc.cc:205] Tried to move back new op lower bound from 7317168868022640640 to 7317168867824267264. Current Snapshot: MvccSnapshot[applied={T|T < 7317168868022640640}]
I20260811 03:16:20.704099 32414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:20.704381 32414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:20.708004 32414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1248d6a014144d59bf3b6d51aeffe775: Sending DeleteTablet for 3 replicas of tablet 49badcb4a2db47e0a04103a74a3788a3
I20260811 03:16:20.709263 379 tablet_service.cc:1514] Processing DeleteTablet for tablet 49badcb4a2db47e0a04103a74a3788a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:20 UTC) from {username='slave'} at 127.0.0.1:39730
I20260811 03:16:20.709378 32704 tablet_service.cc:1514] Processing DeleteTablet for tablet 49badcb4a2db47e0a04103a74a3788a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:20 UTC) from {username='slave'} at 127.0.0.1:41650
I20260811 03:16:20.709439 32551 tablet_service.cc:1514] Processing DeleteTablet for tablet 49badcb4a2db47e0a04103a74a3788a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:20 UTC) from {username='slave'} at 127.0.0.1:34310
I20260811 03:16:20.711352 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 32483
I20260811 03:16:20.723621 498 tablet_replica.cc:330] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: stopping tablet replica
I20260811 03:16:20.723850 499 tablet_replica.cc:330] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: stopping tablet replica
I20260811 03:16:20.724303 499 raft_consensus.cc:2229] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:20.724311 498 raft_consensus.cc:2229] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:20.724938 499 raft_consensus.cc:2258] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:20.724977 498 raft_consensus.cc:2258] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:20.726544 497 tablet_replica.cc:330] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: stopping tablet replica
I20260811 03:16:20.726960 497 raft_consensus.cc:2229] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:16:20.727658 497 raft_consensus.cc:2258] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:20.727810 498 ts_tablet_manager.cc:1905] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:20.728417 499 ts_tablet_manager.cc:1905] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:20.730098 497 ts_tablet_manager.cc:1905] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:20.735445 499 ts_tablet_manager.cc:1918] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:20.735682 499 log.cc:1196] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/49badcb4a2db47e0a04103a74a3788a3
I20260811 03:16:20.736433 499 ts_tablet_manager.cc:1939] T 49badcb4a2db47e0a04103a74a3788a3 P 58c95847ec024adab215b07120dbfe56: Deleting consensus metadata
I20260811 03:16:20.737581 497 ts_tablet_manager.cc:1918] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:20.737813 497 log.cc:1196] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/49badcb4a2db47e0a04103a74a3788a3
I20260811 03:16:20.738519 497 ts_tablet_manager.cc:1939] T 49badcb4a2db47e0a04103a74a3788a3 P b0fa7723416a49448f20e68ed84a28fd: Deleting consensus metadata
I20260811 03:16:20.738551 32399 catalog_manager.cc:4841] TS 58c95847ec024adab215b07120dbfe56 (127.11.60.2:42921): tablet 49badcb4a2db47e0a04103a74a3788a3 (table test-table [id=38cdffac9e6847d19f7250de37f9585b]) successfully deleted
I20260811 03:16:20.739918 498 ts_tablet_manager.cc:1918] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:20.740188 498 log.cc:1196] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/49badcb4a2db47e0a04103a74a3788a3
I20260811 03:16:20.740545 32398 catalog_manager.cc:4841] TS b0fa7723416a49448f20e68ed84a28fd (127.11.60.3:37841): tablet 49badcb4a2db47e0a04103a74a3788a3 (table test-table [id=38cdffac9e6847d19f7250de37f9585b]) successfully deleted
I20260811 03:16:20.741369 498 ts_tablet_manager.cc:1939] T 49badcb4a2db47e0a04103a74a3788a3 P de8d9ecfdd904ce38a4a165508f8db54: Deleting consensus metadata
I20260811 03:16:20.741475 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.712819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34318 (local address 127.11.60.1:37223)
0811 03:16:20.713017 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.713023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.725833 (+ 12810us) server_negotiation.cc:407] Connection header received
0811 03:16:20.726062 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.726068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.726170 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.726280 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:20.728472 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.729596 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.734995 (+ 5399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.735724 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.735859 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.739495 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:20.739508 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:20.739528 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:20.740383 (+ 855us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:20.740461 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:20.740494 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:20.740603 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.740896 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.741042 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":67}
I20260811 03:16:20.742853 32623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:20.743330 32399 catalog_manager.cc:4841] TS de8d9ecfdd904ce38a4a165508f8db54 (127.11.60.1:37223): tablet 49badcb4a2db47e0a04103a74a3788a3 (table test-table [id=38cdffac9e6847d19f7250de37f9585b]) successfully deleted
I20260811 03:16:20.833966 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 32483
I20260811 03:16:20.861642 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 32634
I20260811 03:16:20.869980 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.862584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.11.60.2:42921)
0811 03:16:20.862788 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.862793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.862833 (+ 40us) server_negotiation.cc:407] Connection header received
0811 03:16:20.863051 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.863055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.863170 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.863263 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:20.863752 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.864602 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.866792 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.867246 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.867352 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.868562 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:20.868571 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:20.868585 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:20.869177 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:20.869224 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:20.869244 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:20.869339 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.869498 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.869589 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 03:16:20.870759 303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:20.962560 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 32634
I20260811 03:16:20.988248 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 312
I20260811 03:16:20.997660 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:20.989301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.11.60.3:37841)
0811 03:16:20.989460 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:20.989464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:20.989486 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:16:20.989620 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:20.989624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:20.989699 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:20.989786 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:20.990510 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.991337 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.993369 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:20.993995 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:20.994145 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:20.996112 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:20.996121 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:20.996136 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:20.996838 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:20.996906 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:20.996927 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:20.997063 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:20.997201 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 03:16:20.997297 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 03:16:20.998461 447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:21.121364 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 312
I20260811 03:16:21.145062 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 32380
I20260811 03:16:21.146273 32451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:21.258409 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 32380
2026-08-11T03:16:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2920 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:21 dist-test-slave-tm1g krb5kdc[509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:21 dist-test-slave-tm1g krb5kdc[509](info): set up 2 sockets
Aug 11 03:16:21 dist-test-slave-tm1g krb5kdc[509](info): commencing operation
krb5kdc: starting...
W20260811 03:16:22.479187 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:22 dist-test-slave-tm1g krb5kdc[509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:16:22.805786 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:32973
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:32973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:23.158773 522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:23.159133 522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:23.159291 522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:23.166739 522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:23.166891 522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:23.166988 522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:23.167099 522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:23.167176 522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:23.177302 522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:32973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:32973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:23.178774 522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:23.180177 522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:23.192561 530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:23.192797 528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:23.193012 527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:23.193980 522 server_base.cc:1034] running on GCE node
I20260811 03:16:23.194919 522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:23.196374 522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:23.197584 522 hybrid_clock.cc:648] HybridClock initialized: now 1786418183197544 us; error 27 us; skew 500 ppm
I20260811 03:16:23.197958 522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:23 dist-test-slave-tm1g krb5kdc[509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418183, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:23.201190 522 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:16:23.202817 522 webserver.cc:460] Webserver started at http://127.11.60.62:46549/ using document root <none> and password file <none>
I20260811 03:16:23.203526 522 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:23.203634 522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:23.203890 522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:23.207105 522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "0362c5194af04737bf5cf7bd3b09d453"
format_stamp: "Formatted at 2026-08-11 03:16:23 on dist-test-slave-tm1g"
server_key: "827145a6f830f3ab1f520080c761610f"
server_key_iv: "b69cee765e40a31bc19dd203fbd32581"
server_key_version: "encryptionkey@0"
I20260811 03:16:23.207894 522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "0362c5194af04737bf5cf7bd3b09d453"
format_stamp: "Formatted at 2026-08-11 03:16:23 on dist-test-slave-tm1g"
server_key: "827145a6f830f3ab1f520080c761610f"
server_key_iv: "b69cee765e40a31bc19dd203fbd32581"
server_key_version: "encryptionkey@0"
I20260811 03:16:23.213353 522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:16:23.217006 539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:23.218533 522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:16:23.218735 522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "0362c5194af04737bf5cf7bd3b09d453"
format_stamp: "Formatted at 2026-08-11 03:16:23 on dist-test-slave-tm1g"
server_key: "827145a6f830f3ab1f520080c761610f"
server_key_iv: "b69cee765e40a31bc19dd203fbd32581"
server_key_version: "encryptionkey@0"
I20260811 03:16:23.218940 522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:23.249130 522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:23.253850 522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:23.254196 522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:23.274861 522 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:32973
I20260811 03:16:23.274894 595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:32973 every 8 connection(s)
I20260811 03:16:23.276319 522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:23.277788 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 522
I20260811 03:16:23.278002 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:23.278612 11504 external_mini_cluster.cc:1428] Setting key a85b6f8cd21ad98135782aaaed4b4b25
I20260811 03:16:23.281356 596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:23.292415 596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453: Bootstrap starting.
Aug 11 03:16:23 dist-test-slave-tm1g krb5kdc[509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:23.295970 596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:23.297116 596 log.cc:824] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:23.301277 596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453: No bootstrap required, opened a new log
I20260811 03:16:23.302809 599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:23.280795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.11.60.62:32973)
0811 03:16:23.281690 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:23.281702 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:23.281745 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:16:23.283554 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:23.283576 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:23.283991 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:23.284414 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:23.286024 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:23.287502 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:23.288832 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:23.289560 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:23.289777 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:23.294031 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:23.294039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:23.294052 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:23.294092 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:23.297695 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:23.298757 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:23.298764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:23.298771 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:23.299032 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:23.299459 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:23.299464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:23.299467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:23.300189 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:23.300499 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:23.300900 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 03:16:23.301366 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":367,"threads_started":1}
I20260811 03:16:23.309989 596 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0362c5194af04737bf5cf7bd3b09d453" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 32973 } }
I20260811 03:16:23.310446 596 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:23.310557 596 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0362c5194af04737bf5cf7bd3b09d453, State: Initialized, Role: FOLLOWER
I20260811 03:16:23.311259 596 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0362c5194af04737bf5cf7bd3b09d453" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 32973 } }
I20260811 03:16:23.311594 596 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:23.311733 596 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:23.311923 596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:23.314162 596 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0362c5194af04737bf5cf7bd3b09d453" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 32973 } }
I20260811 03:16:23.314718 596 leader_election.cc:302] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0362c5194af04737bf5cf7bd3b09d453; no voters:
I20260811 03:16:23.315368 596 leader_election.cc:288] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:23.315574 601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:23.316358 601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [term 1 LEADER]: Becoming Leader. State: Replica: 0362c5194af04737bf5cf7bd3b09d453, State: Running, Role: LEADER
I20260811 03:16:23.316972 601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0362c5194af04737bf5cf7bd3b09d453" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 32973 } }
I20260811 03:16:23.317723 596 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:23.320354 603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0362c5194af04737bf5cf7bd3b09d453. Latest consensus state: current_term: 1 leader_uuid: "0362c5194af04737bf5cf7bd3b09d453" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0362c5194af04737bf5cf7bd3b09d453" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 32973 } } }
I20260811 03:16:23.320996 603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:23.320544 602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0362c5194af04737bf5cf7bd3b09d453" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0362c5194af04737bf5cf7bd3b09d453" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 32973 } } }
I20260811 03:16:23.326820 602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:23.328001 609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:23.332517 609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:23.340911 609 catalog_manager.cc:1347] Generated new cluster ID: 77db10ba558c405ebc08e44b2bdf5269
I20260811 03:16:23.341043 609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:23.357514 609 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:23.358930 609 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:23.368185 609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453: Generated new TSK 0
I20260811 03:16:23.369040 609 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.60.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
Entry for principal kudu/127.11.60.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab.
I20260811 03:16:23.447185 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:32973
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:23.825188 625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:23.825517 625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:23.825675 625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:23.833105 625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:23.833211 625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:23.833431 625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:16:23.843412 625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.1.keytab
--principal=kudu/127.11.60.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:32973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:23.844738 625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:23.846133 625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:23.859870 630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:23.859787 633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:23.859879 631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:23.860807 625 server_base.cc:1034] running on GCE node
I20260811 03:16:23.861439 625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:23.862383 625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:23.863620 625 hybrid_clock.cc:648] HybridClock initialized: now 1786418183863552 us; error 52 us; skew 500 ppm
I20260811 03:16:23.863919 625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:23 dist-test-slave-tm1g krb5kdc[509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418183, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:23.872471 625 init.cc:377] Logged in from keytab as kudu/127.11.60.1@KRBTEST.COM (short username kudu)
I20260811 03:16:23.874030 625 webserver.cc:460] Webserver started at http://127.11.60.1:42105/ using document root <none> and password file <none>
I20260811 03:16:23.874804 625 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:23.874908 625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:23.875187 625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:23.878281 625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "1ad63dd167074903bb0cadfa4ce059dd"
format_stamp: "Formatted at 2026-08-11 03:16:23 on dist-test-slave-tm1g"
server_key: "fa97f267d4512d96439e22886dae5135"
server_key_iv: "077f391dd3ecfc3b6e260ff9e794e815"
server_key_version: "encryptionkey@0"
I20260811 03:16:23.879106 625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "1ad63dd167074903bb0cadfa4ce059dd"
format_stamp: "Formatted at 2026-08-11 03:16:23 on dist-test-slave-tm1g"
server_key: "fa97f267d4512d96439e22886dae5135"
server_key_iv: "077f391dd3ecfc3b6e260ff9e794e815"
server_key_version: "encryptionkey@0"
I20260811 03:16:23.884575 625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:16:23.888130 640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:23.889575 625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:16:23.889716 625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "1ad63dd167074903bb0cadfa4ce059dd"
format_stamp: "Formatted at 2026-08-11 03:16:23 on dist-test-slave-tm1g"
server_key: "fa97f267d4512d96439e22886dae5135"
server_key_iv: "077f391dd3ecfc3b6e260ff9e794e815"
server_key_version: "encryptionkey@0"
I20260811 03:16:23.889894 625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:23.908813 625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:23.913166 625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:23.913414 625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:23.914352 625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:23.915693 625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:23.915766 625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:23.915853 625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:23.915896 625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:23.952893 625 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:44313
I20260811 03:16:23.952921 766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:44313 every 8 connection(s)
I20260811 03:16:23.954334 625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:16:23.963223 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 625
I20260811 03:16:23.963440 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:16:23.963893 11504 external_mini_cluster.cc:1428] Setting key d0bdd84dfe7b07bc69b408a247847b1f
Aug 11 03:16:23 dist-test-slave-tm1g krb5kdc[509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418183, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.1@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:23.975849 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:23.957323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:51681 (local address 127.11.60.62:32973)
0811 03:16:23.957864 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:23.957869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:23.958860 (+ 991us) server_negotiation.cc:407] Connection header received
0811 03:16:23.960772 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:23.960778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:23.960891 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:23.961025 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:23.962645 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:23.963455 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:23.965328 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:23.965755 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:23.965914 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:23.969858 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:23.969864 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:23.969867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:23.969900 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:23.972624 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:23.973388 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:23.973393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:23.973399 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:23.973524 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:23.973989 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:23.973996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:23.974003 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:23.974280 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:23.974505 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:23.975227 (+ 722us) server_negotiation.cc:299] Negotiation successful
0811 03:16:23.975476 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":304,"threads_started":1}
I20260811 03:16:23.976867 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:23.957823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:32973 (local address 127.11.60.1:51681)
0811 03:16:23.958680 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:23.958727 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:23.960489 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:23.961274 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:23.961283 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:23.961805 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:23.962463 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:23.962483 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:23.963653 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:23.963657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:23.965161 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:23.965174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:23.966013 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:23.966023 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:23.966263 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:23.967071 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:16:23.967100 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:23.969621 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:23.972778 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:23.972789 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:23.972815 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:23.973246 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:23.973648 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:23.973653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:23.973657 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:23.973862 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:23.974463 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:23.974472 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:23.974838 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 03:16:23.975118 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 03:16:23.975667 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":311,"threads_started":1}
I20260811 03:16:23.978637 767 heartbeater.cc:347] Connected to a master server at 127.11.60.62:32973
I20260811 03:16:23.978972 767 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:23.979888 767 heartbeater.cc:511] Master 127.11.60.62:32973 requested a full tablet report, sending...
I20260811 03:16:23.982479 558 ts_manager.cc:194] Registered new tserver with Master: 1ad63dd167074903bb0cadfa4ce059dd (127.11.60.1:44313)
I20260811 03:16:23.984694 558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.1@KRBTEST.COM'} at 127.11.60.1:51681
WARNING: no policy specified for kudu/127.11.60.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
Entry for principal kudu/127.11.60.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab.
I20260811 03:16:24.040263 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:32973
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:24.397373 774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:24.397702 774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:24.397869 774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:24.405978 774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:24.406170 774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:24.406435 774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:16:24.417218 774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.2.keytab
--principal=kudu/127.11.60.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:32973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:24.418720 774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:24.420125 774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:24.434029 780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:24.434188 779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:24.434118 782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:24.435355 774 server_base.cc:1034] running on GCE node
I20260811 03:16:24.435920 774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:24.436882 774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:24.438110 774 hybrid_clock.cc:648] HybridClock initialized: now 1786418184438067 us; error 27 us; skew 500 ppm
I20260811 03:16:24.438474 774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:24 dist-test-slave-tm1g krb5kdc[509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418184, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:24.441633 774 init.cc:377] Logged in from keytab as kudu/127.11.60.2@KRBTEST.COM (short username kudu)
I20260811 03:16:24.443140 774 webserver.cc:460] Webserver started at http://127.11.60.2:40481/ using document root <none> and password file <none>
I20260811 03:16:24.443795 774 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:24.443895 774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:24.444167 774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:24.447149 774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8"
format_stamp: "Formatted at 2026-08-11 03:16:24 on dist-test-slave-tm1g"
server_key: "a6704a337907458fa59c10e5a6b15e8f"
server_key_iv: "32c3fa539ad25ee6502305f3b14aa633"
server_key_version: "encryptionkey@0"
I20260811 03:16:24.447901 774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8"
format_stamp: "Formatted at 2026-08-11 03:16:24 on dist-test-slave-tm1g"
server_key: "a6704a337907458fa59c10e5a6b15e8f"
server_key_iv: "32c3fa539ad25ee6502305f3b14aa633"
server_key_version: "encryptionkey@0"
I20260811 03:16:24.453199 774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:16:24.456897 789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:24.458214 774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 03:16:24.458410 774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8"
format_stamp: "Formatted at 2026-08-11 03:16:24 on dist-test-slave-tm1g"
server_key: "a6704a337907458fa59c10e5a6b15e8f"
server_key_iv: "32c3fa539ad25ee6502305f3b14aa633"
server_key_version: "encryptionkey@0"
I20260811 03:16:24.458644 774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:24.469743 774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:24.473670 774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:24.473910 774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:24.474915 774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:24.476168 774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:24.476243 774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:24.476388 774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:24.476456 774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:24.511475 774 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:42385
I20260811 03:16:24.511571 911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:42385 every 8 connection(s)
I20260811 03:16:24.512883 774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:16:24.522722 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 774
I20260811 03:16:24.522935 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:16:24.523547 11504 external_mini_cluster.cc:1428] Setting key 8c5a6019532d6fa58fb63acf8c9b74a5
Aug 11 03:16:24 dist-test-slave-tm1g krb5kdc[509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418184, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.2@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:24.536278 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:24.515796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:35263 (local address 127.11.60.62:32973)
0811 03:16:24.516281 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:24.516286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:24.517369 (+ 1083us) server_negotiation.cc:407] Connection header received
0811 03:16:24.519404 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:24.519409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:24.519520 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:24.519639 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:24.521163 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:24.521900 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:24.524352 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:24.524972 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:24.525106 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:24.528888 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:24.528898 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:24.528905 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:24.528955 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:24.532670 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:24.533528 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:24.533537 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:24.533543 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:24.533711 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:24.534195 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:24.534203 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:24.534208 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:24.534647 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:24.534804 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:24.535526 (+ 722us) server_negotiation.cc:299] Negotiation successful
0811 03:16:24.535800 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":249,"threads_started":1}
I20260811 03:16:24.537145 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:24.516455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:32973 (local address 127.11.60.2:35263)
0811 03:16:24.517204 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:24.517248 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:24.519123 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:24.519875 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:24.519884 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:24.520422 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:24.520988 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:24.521005 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:24.522061 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:24.522065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:24.524200 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:24.524219 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:24.525232 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:24.525238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:24.525543 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:24.526383 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:16:24.526415 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:24.528599 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:24.532834 (+ 4235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:24.532842 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:24.532859 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:24.533384 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:24.533856 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:24.533862 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:24.533866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:24.534066 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:24.534781 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:24.534788 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:24.535127 (+ 339us) client_negotiation.cc:769] Sending connection context
0811 03:16:24.535462 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 03:16:24.535918 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":304,"threads_started":1}
I20260811 03:16:24.538702 912 heartbeater.cc:347] Connected to a master server at 127.11.60.62:32973
I20260811 03:16:24.539052 912 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:24.539816 912 heartbeater.cc:511] Master 127.11.60.62:32973 requested a full tablet report, sending...
I20260811 03:16:24.541396 558 ts_manager.cc:194] Registered new tserver with Master: 7dcba28ea8374b79a4c1f34dfb40ffe8 (127.11.60.2:42385)
I20260811 03:16:24.542721 558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.2@KRBTEST.COM'} at 127.11.60.2:35263
WARNING: no policy specified for kudu/127.11.60.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
Entry for principal kudu/127.11.60.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab.
I20260811 03:16:24.598822 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:32973
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:24.944803 921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:24.945104 921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:24.945276 921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:24.952927 921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:24.953047 921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:24.953227 921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:16:24.963886 921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.3.keytab
--principal=kudu/127.11.60.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:32973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:24.965291 921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:24.966837 921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:24.980631 927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:24.980671 930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:24.980590 928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:24.981413 921 server_base.cc:1034] running on GCE node
I20260811 03:16:24.982043 921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:24.983139 921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:24.984356 921 hybrid_clock.cc:648] HybridClock initialized: now 1786418184984321 us; error 35 us; skew 500 ppm
I20260811 03:16:24.984730 921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:24 dist-test-slave-tm1g krb5kdc[509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418184, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:24.988204 921 init.cc:377] Logged in from keytab as kudu/127.11.60.3@KRBTEST.COM (short username kudu)
I20260811 03:16:24.988883 767 heartbeater.cc:503] Master 127.11.60.62:32973 was elected leader, sending a full tablet report...
I20260811 03:16:24.990650 921 webserver.cc:460] Webserver started at http://127.11.60.3:45053/ using document root <none> and password file <none>
I20260811 03:16:24.991513 921 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:24.991659 921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:24.992002 921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:24.995200 921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "6549b657bcde4e29b2c2b20a833ed42b"
format_stamp: "Formatted at 2026-08-11 03:16:24 on dist-test-slave-tm1g"
server_key: "302098d693cbb6fddb6f778b29effb56"
server_key_iv: "12590ba4364b81cec03644ab8e9d6fc4"
server_key_version: "encryptionkey@0"
I20260811 03:16:24.996038 921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "6549b657bcde4e29b2c2b20a833ed42b"
format_stamp: "Formatted at 2026-08-11 03:16:24 on dist-test-slave-tm1g"
server_key: "302098d693cbb6fddb6f778b29effb56"
server_key_iv: "12590ba4364b81cec03644ab8e9d6fc4"
server_key_version: "encryptionkey@0"
I20260811 03:16:25.001534 921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:16:25.005524 937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:25.006982 921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:16:25.007159 921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "6549b657bcde4e29b2c2b20a833ed42b"
format_stamp: "Formatted at 2026-08-11 03:16:24 on dist-test-slave-tm1g"
server_key: "302098d693cbb6fddb6f778b29effb56"
server_key_iv: "12590ba4364b81cec03644ab8e9d6fc4"
server_key_version: "encryptionkey@0"
I20260811 03:16:25.007375 921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:25.025764 921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:25.030911 921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:25.031224 921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:25.032419 921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:25.033852 921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:25.033959 921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:25.034116 921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:25.034191 921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:25.067669 921 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:41559
I20260811 03:16:25.067714 1064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:41559 every 8 connection(s)
I20260811 03:16:25.069039 921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:16:25.071098 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 921
I20260811 03:16:25.071305 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:16:25.071848 11504 external_mini_cluster.cc:1428] Setting key 1a0ab2fcb9e19cd7f1455da103c5d17c
Aug 11 03:16:25 dist-test-slave-tm1g krb5kdc[509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418184, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.3@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:25.091187 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.071803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:35549 (local address 127.11.60.62:32973)
0811 03:16:25.072694 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.072701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.073890 (+ 1189us) server_negotiation.cc:407] Connection header received
0811 03:16:25.076485 (+ 2595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.076490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.076566 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.076662 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:25.078179 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.079032 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.080647 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.081064 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.081163 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.084789 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:25.084793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:25.084797 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:25.084833 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:25.088150 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:25.088890 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:25.088894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:25.088897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:25.089034 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:25.089526 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:25.089529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:25.089531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:25.089808 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:25.089938 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.090730 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.090902 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":600,"threads_started":1}
I20260811 03:16:25.092144 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.072489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:32973 (local address 127.11.60.3:35549)
0811 03:16:25.073719 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:25.073775 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:25.076226 (+ 2451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:25.076891 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:25.076899 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:25.077456 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:25.078029 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.078047 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.079177 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.079181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.080508 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.080521 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.081261 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.081265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.081506 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.082165 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:16:25.082192 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:25.084563 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:25.088298 (+ 3735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:25.088306 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:25.088328 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:25.088756 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:25.089162 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:16:25.089166 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:16:25.089169 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:16:25.089391 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:16:25.089923 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:25.089930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:25.090252 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 03:16:25.090644 (+ 392us) client_negotiation.cc:240] Negotiation successful
0811 03:16:25.091040 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"thread_start_us":718,"threads_started":1}
I20260811 03:16:25.093588 1065 heartbeater.cc:347] Connected to a master server at 127.11.60.62:32973
I20260811 03:16:25.093920 1065 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:25.094741 1065 heartbeater.cc:511] Master 127.11.60.62:32973 requested a full tablet report, sending...
I20260811 03:16:25.096171 558 ts_manager.cc:194] Registered new tserver with Master: 6549b657bcde4e29b2c2b20a833ed42b (127.11.60.3:41559)
I20260811 03:16:25.096999 558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.60.3@KRBTEST.COM'} at 127.11.60.3:35549
I20260811 03:16:25.100126 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:16:25.117539 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.104288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48246 (local address 127.11.60.62:32973)
0811 03:16:25.104451 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.104455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.104954 (+ 499us) server_negotiation.cc:407] Connection header received
0811 03:16:25.105134 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.105138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.105196 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.105277 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:25.106747 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.107499 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.108765 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.109163 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.109262 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.111953 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:25.111958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:25.111962 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:25.111995 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:25.114681 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:25.115351 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:25.115355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:25.115357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:25.115505 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:25.115975 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:25.115979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:25.115981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:25.116354 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:25.116596 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.116991 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.117176 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 03:16:25.121847 558 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:16:25.129006 558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:16:25.153327 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.141647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:42385 (local address 127.0.0.1:51892)
0811 03:16:25.143710 (+ 2063us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:25.143729 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:25.143876 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:25.144903 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:25.144913 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:25.144944 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:25.145400 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.145414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.147496 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.147501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.150035 (+ 2534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.150050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.152276 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.152284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.152495 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.152551 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 03:16:25.152676 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 03:16:25.152898 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1933,"thread_start_us":356,"threads_started":1}
I20260811 03:16:25.154273 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.141946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.11.60.2:42385)
0811 03:16:25.143430 (+ 1484us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.143442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.144008 (+ 566us) server_negotiation.cc:407] Connection header received
0811 03:16:25.144118 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.144128 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.144407 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.144601 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:25.145627 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.147331 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.150232 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.151753 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.151911 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.152128 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.153747 (+ 1619us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.153947 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1295,"thread_start_us":1162,"threads_started":1}
I20260811 03:16:25.155809 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.141077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:44313 (local address 127.0.0.1:35752)
0811 03:16:25.142499 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:25.142562 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:25.142772 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:25.145751 (+ 2979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:25.145757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:25.145785 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:25.146212 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.146224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.149530 (+ 3306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.149536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.152956 (+ 3420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.152969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.154984 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.154990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.155195 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.155234 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 03:16:25.155310 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 03:16:25.155450 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"thread_start_us":2342,"threads_started":1}
I20260811 03:16:25.158141 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.141201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.11.60.1:44313)
0811 03:16:25.145130 (+ 3929us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.145139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.145166 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:16:25.145276 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.145284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.145530 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.146448 (+ 918us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:25.146728 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.149361 (+ 2633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.153181 (+ 3820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.154730 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.156061 (+ 1331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.156354 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.156454 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.156638 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2594,"thread_start_us":2423,"threads_started":1}
I20260811 03:16:25.159458 840 tablet_service.cc:1467] Processing CreateTablet for tablet f6de0567e9894f54a4c5ff906c6fe7dd (DEFAULT_TABLE table=test-table [id=bea25422c2034bd8a06a5f095c30f0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:25.160977 840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6de0567e9894f54a4c5ff906c6fe7dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:25.161420 693 tablet_service.cc:1467] Processing CreateTablet for tablet f6de0567e9894f54a4c5ff906c6fe7dd (DEFAULT_TABLE table=test-table [id=bea25422c2034bd8a06a5f095c30f0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:25.162943 693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6de0567e9894f54a4c5ff906c6fe7dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:25.165252 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.143847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:41559 (local address 127.0.0.1:40686)
0811 03:16:25.154754 (+ 10907us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:25.154775 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:25.154898 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:25.158544 (+ 3646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:25.158551 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:25.158580 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:25.158730 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.158740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.160748 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.160753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.162631 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.162647 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.164491 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.164496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.164691 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.164723 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 03:16:25.164813 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 03:16:25.164965 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10812,"thread_start_us":14368,"threads_started":1}
I20260811 03:16:25.165452 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.144365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.11.60.3:41559)
0811 03:16:25.156709 (+ 12344us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.156723 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.156766 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:16:25.156907 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.156921 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.157250 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.157509 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:25.158929 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.160580 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.162841 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.164304 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.164468 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.164709 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.164907 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.165074 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12075,"thread_start_us":13403,"threads_started":1}
I20260811 03:16:25.169401 994 tablet_service.cc:1467] Processing CreateTablet for tablet f6de0567e9894f54a4c5ff906c6fe7dd (DEFAULT_TABLE table=test-table [id=bea25422c2034bd8a06a5f095c30f0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:25.171031 994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6de0567e9894f54a4c5ff906c6fe7dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:25.175712 1087 tablet_bootstrap.cc:492] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: Bootstrap starting.
I20260811 03:16:25.177708 1088 tablet_bootstrap.cc:492] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: Bootstrap starting.
I20260811 03:16:25.179633 1087 tablet_bootstrap.cc:654] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:25.180996 1087 log.cc:824] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:25.181319 1088 tablet_bootstrap.cc:654] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:25.182744 1088 log.cc:824] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:25.185402 1087 tablet_bootstrap.cc:492] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: No bootstrap required, opened a new log
I20260811 03:16:25.185702 1091 tablet_bootstrap.cc:492] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: Bootstrap starting.
I20260811 03:16:25.185745 1087 ts_tablet_manager.cc:1397] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 03:16:25.187215 1088 tablet_bootstrap.cc:492] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: No bootstrap required, opened a new log
I20260811 03:16:25.187486 1088 ts_tablet_manager.cc:1397] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 03:16:25.189288 1091 tablet_bootstrap.cc:654] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:25.190605 1091 log.cc:824] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:25.193866 1091 tablet_bootstrap.cc:492] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: No bootstrap required, opened a new log
I20260811 03:16:25.194139 1091 ts_tablet_manager.cc:1397] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 03:16:25.196815 1087 raft_consensus.cc:348] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 } } peers { permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 } } peers { permanent_uuid: "6549b657bcde4e29b2c2b20a833ed42b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41559 } }
I20260811 03:16:25.197026 1088 raft_consensus.cc:348] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 } } peers { permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 } } peers { permanent_uuid: "6549b657bcde4e29b2c2b20a833ed42b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41559 } }
I20260811 03:16:25.197299 1087 raft_consensus.cc:374] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:25.197402 1087 raft_consensus.cc:729] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad63dd167074903bb0cadfa4ce059dd, State: Initialized, Role: FOLLOWER
I20260811 03:16:25.197456 1088 raft_consensus.cc:374] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:25.197604 1088 raft_consensus.cc:729] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcba28ea8374b79a4c1f34dfb40ffe8, State: Initialized, Role: FOLLOWER
I20260811 03:16:25.198062 1087 consensus_queue.cc:260] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 } } peers { permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 } } peers { permanent_uuid: "6549b657bcde4e29b2c2b20a833ed42b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41559 } }
I20260811 03:16:25.198258 1088 consensus_queue.cc:260] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 } } peers { permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 } } peers { permanent_uuid: "6549b657bcde4e29b2c2b20a833ed42b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41559 } }
I20260811 03:16:25.199525 912 heartbeater.cc:503] Master 127.11.60.62:32973 was elected leader, sending a full tablet report...
I20260811 03:16:25.200234 1088 ts_tablet_manager.cc:1428] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260811 03:16:25.201915 1087 ts_tablet_manager.cc:1428] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260811 03:16:25.206010 1091 raft_consensus.cc:348] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 } } peers { permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 } } peers { permanent_uuid: "6549b657bcde4e29b2c2b20a833ed42b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41559 } }
I20260811 03:16:25.206569 1091 raft_consensus.cc:374] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:25.206693 1091 raft_consensus.cc:729] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6549b657bcde4e29b2c2b20a833ed42b, State: Initialized, Role: FOLLOWER
W20260811 03:16:25.206961 768 tablet.cc:2369] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:16:25.207545 1091 consensus_queue.cc:260] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 } } peers { permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 } } peers { permanent_uuid: "6549b657bcde4e29b2c2b20a833ed42b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41559 } }
I20260811 03:16:25.208637 1065 heartbeater.cc:503] Master 127.11.60.62:32973 was elected leader, sending a full tablet report...
I20260811 03:16:25.209085 1091 ts_tablet_manager.cc:1428] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260811 03:16:25.255987 1096 raft_consensus.cc:482] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:16:25.256366 1096 raft_consensus.cc:504] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 } } peers { permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 } } peers { permanent_uuid: "6549b657bcde4e29b2c2b20a833ed42b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41559 } }
I20260811 03:16:25.258191 1096 leader_election.cc:288] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ad63dd167074903bb0cadfa4ce059dd (127.11.60.1:44313), 7dcba28ea8374b79a4c1f34dfb40ffe8 (127.11.60.2:42385)
I20260811 03:16:25.264624 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.258752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:47537 (local address 127.11.60.1:44313)
0811 03:16:25.258978 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.258983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.259325 (+ 342us) server_negotiation.cc:407] Connection header received
0811 03:16:25.259554 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.259561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.259629 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.259731 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:25.260142 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.261011 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.262856 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.263713 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.263855 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.263962 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.264245 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.264372 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 03:16:25.264618 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.258734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:44313 (local address 127.11.60.3:47537)
0811 03:16:25.259205 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:25.259237 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:25.259386 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:25.259753 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:25.259758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:25.259786 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:25.260000 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.260009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.261153 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.261158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.262695 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.262707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.263879 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.263887 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.264081 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.264114 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 03:16:25.264213 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 03:16:25.264341 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
W20260811 03:16:25.265233 914 tablet.cc:2369] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:16:25.266013 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.258853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:42385 (local address 127.11.60.3:37625)
0811 03:16:25.259687 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:25.259702 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:25.259873 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:25.260217 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:25.260222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:25.260244 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:25.260585 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.260594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.261645 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.261650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.263447 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:25.263463 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.265197 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.265206 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:25.265410 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.265456 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 03:16:25.265558 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 03:16:25.265687 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":490,"threads_started":1}
I20260811 03:16:25.266031 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.258711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:37625 (local address 127.11.60.2:42385)
0811 03:16:25.258952 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.258960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.259799 (+ 839us) server_negotiation.cc:407] Connection header received
0811 03:16:25.260043 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.260048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.260108 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.260204 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:25.260750 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.261489 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.263633 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.264805 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.264943 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.265050 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.265608 (+ 558us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.265749 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 03:16:25.265957 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6de0567e9894f54a4c5ff906c6fe7dd" candidate_uuid: "6549b657bcde4e29b2c2b20a833ed42b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad63dd167074903bb0cadfa4ce059dd" is_pre_election: true
I20260811 03:16:25.266582 714 raft_consensus.cc:2454] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6549b657bcde4e29b2c2b20a833ed42b in term 0.
I20260811 03:16:25.267233 940 leader_election.cc:302] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ad63dd167074903bb0cadfa4ce059dd, 6549b657bcde4e29b2c2b20a833ed42b; no voters:
I20260811 03:16:25.267619 1096 raft_consensus.cc:2781] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:16:25.267746 1096 raft_consensus.cc:482] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:16:25.267869 1096 raft_consensus.cc:3037] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:25.267643 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6de0567e9894f54a4c5ff906c6fe7dd" candidate_uuid: "6549b657bcde4e29b2c2b20a833ed42b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" is_pre_election: true
I20260811 03:16:25.268179 861 raft_consensus.cc:2454] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6549b657bcde4e29b2c2b20a833ed42b in term 0.
I20260811 03:16:25.270188 1096 raft_consensus.cc:504] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 } } peers { permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 } } peers { permanent_uuid: "6549b657bcde4e29b2c2b20a833ed42b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41559 } }
I20260811 03:16:25.270896 1096 leader_election.cc:288] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [CANDIDATE]: Term 1 election: Requested vote from peers 1ad63dd167074903bb0cadfa4ce059dd (127.11.60.1:44313), 7dcba28ea8374b79a4c1f34dfb40ffe8 (127.11.60.2:42385)
I20260811 03:16:25.271334 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6de0567e9894f54a4c5ff906c6fe7dd" candidate_uuid: "6549b657bcde4e29b2c2b20a833ed42b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad63dd167074903bb0cadfa4ce059dd"
I20260811 03:16:25.271447 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6de0567e9894f54a4c5ff906c6fe7dd" candidate_uuid: "6549b657bcde4e29b2c2b20a833ed42b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8"
I20260811 03:16:25.271559 714 raft_consensus.cc:3037] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:25.271684 861 raft_consensus.cc:3037] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:25.273857 714 raft_consensus.cc:2454] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6549b657bcde4e29b2c2b20a833ed42b in term 1.
I20260811 03:16:25.273931 861 raft_consensus.cc:2454] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6549b657bcde4e29b2c2b20a833ed42b in term 1.
I20260811 03:16:25.274245 940 leader_election.cc:302] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ad63dd167074903bb0cadfa4ce059dd, 6549b657bcde4e29b2c2b20a833ed42b; no voters:
I20260811 03:16:25.274596 1096 raft_consensus.cc:2781] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:25.275133 1096 raft_consensus.cc:686] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 1 LEADER]: Becoming Leader. State: Replica: 6549b657bcde4e29b2c2b20a833ed42b, State: Running, Role: LEADER
I20260811 03:16:25.275705 1096 consensus_queue.cc:237] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 } } peers { permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 } } peers { permanent_uuid: "6549b657bcde4e29b2c2b20a833ed42b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41559 } }
I20260811 03:16:25.280791 557 catalog_manager.cc:5495] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6549b657bcde4e29b2c2b20a833ed42b (127.11.60.3). New cstate: current_term: 1 leader_uuid: "6549b657bcde4e29b2c2b20a833ed42b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6549b657bcde4e29b2c2b20a833ed42b" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 41559 } health_report { overall_health: HEALTHY } } }
I20260811 03:16:25.306845 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.301198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40690 (local address 127.11.60.3:41559)
0811 03:16:25.301468 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.301476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.301586 (+ 110us) server_negotiation.cc:407] Connection header received
0811 03:16:25.301837 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.301841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.301918 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.302076 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:16:25.302549 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.303280 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.304672 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.305076 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.305208 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.305734 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:16:25.306242 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:16:25.306451 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.306502 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.306584 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 03:16:25.315907 714 raft_consensus.cc:1261] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd [term 1 FOLLOWER]: Refusing update from remote peer 6549b657bcde4e29b2c2b20a833ed42b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:25.315912 861 raft_consensus.cc:1261] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8 [term 1 FOLLOWER]: Refusing update from remote peer 6549b657bcde4e29b2c2b20a833ed42b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:25.317176 1098 consensus_queue.cc:1035] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dcba28ea8374b79a4c1f34dfb40ffe8" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:25.317593 1096 consensus_queue.cc:1035] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ad63dd167074903bb0cadfa4ce059dd" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 03:16:25.320695 1066 tablet.cc:2369] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:16:25.330976 1103 mvcc.cc:205] Tried to move back new op lower bound from 7317168887041298432 to 7317168886893813760. Current Snapshot: MvccSnapshot[applied={T|T < 7317168887041298432}]
I20260811 03:16:25.336786 1102 mvcc.cc:205] Tried to move back new op lower bound from 7317168887041298432 to 7317168886893813760. Current Snapshot: MvccSnapshot[applied={T|T < 7317168887041298432}]
I20260811 03:16:25.348029 557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:25.348311 557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:25.351931 557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0362c5194af04737bf5cf7bd3b09d453: Sending DeleteTablet for 3 replicas of tablet f6de0567e9894f54a4c5ff906c6fe7dd
I20260811 03:16:25.353276 840 tablet_service.cc:1514] Processing DeleteTablet for tablet f6de0567e9894f54a4c5ff906c6fe7dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:25 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:51892
I20260811 03:16:25.353505 693 tablet_service.cc:1514] Processing DeleteTablet for tablet f6de0567e9894f54a4c5ff906c6fe7dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:25 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:35752
I20260811 03:16:25.353592 994 tablet_service.cc:1514] Processing DeleteTablet for tablet f6de0567e9894f54a4c5ff906c6fe7dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:25 UTC) from {username='kudu', principal='kudu/127.11.60.62@KRBTEST.COM'} at 127.0.0.1:40686
I20260811 03:16:25.356707 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 625
I20260811 03:16:25.368749 1114 tablet_replica.cc:330] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: stopping tablet replica
I20260811 03:16:25.369400 1113 tablet_replica.cc:330] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: stopping tablet replica
I20260811 03:16:25.369480 1114 raft_consensus.cc:2229] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:25.370081 1113 raft_consensus.cc:2229] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:25.370177 1114 raft_consensus.cc:2258] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:25.370950 1113 raft_consensus.cc:2258] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:25.371145 1115 tablet_replica.cc:330] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: stopping tablet replica
I20260811 03:16:25.373382 1114 ts_tablet_manager.cc:1905] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:25.373690 1113 ts_tablet_manager.cc:1905] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:25.381003 1113 ts_tablet_manager.cc:1918] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:25.381011 1114 ts_tablet_manager.cc:1918] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:25.381192 1114 log.cc:1196] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-0/wal/wals/f6de0567e9894f54a4c5ff906c6fe7dd
I20260811 03:16:25.381351 1113 log.cc:1196] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/f6de0567e9894f54a4c5ff906c6fe7dd
I20260811 03:16:25.381850 1114 ts_tablet_manager.cc:1939] T f6de0567e9894f54a4c5ff906c6fe7dd P 1ad63dd167074903bb0cadfa4ce059dd: Deleting consensus metadata
I20260811 03:16:25.382078 1113 ts_tablet_manager.cc:1939] T f6de0567e9894f54a4c5ff906c6fe7dd P 7dcba28ea8374b79a4c1f34dfb40ffe8: Deleting consensus metadata
I20260811 03:16:25.384116 543 catalog_manager.cc:4841] TS 1ad63dd167074903bb0cadfa4ce059dd (127.11.60.1:44313): tablet f6de0567e9894f54a4c5ff906c6fe7dd (table test-table [id=bea25422c2034bd8a06a5f095c30f0fa]) successfully deleted
I20260811 03:16:25.384146 541 catalog_manager.cc:4841] TS 7dcba28ea8374b79a4c1f34dfb40ffe8 (127.11.60.2:42385): tablet f6de0567e9894f54a4c5ff906c6fe7dd (table test-table [id=bea25422c2034bd8a06a5f095c30f0fa]) successfully deleted
I20260811 03:16:25.384933 1115 raft_consensus.cc:2229] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:16:25.385699 1115 raft_consensus.cc:2258] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 03:16:25 dist-test-slave-tm1g krb5kdc[509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.1@KRBTEST.COM
I20260811 03:16:25.387934 1115 ts_tablet_manager.cc:1905] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:25.394523 1115 ts_tablet_manager.cc:1918] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:25.394731 1115 log.cc:1196] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/f6de0567e9894f54a4c5ff906c6fe7dd
I20260811 03:16:25.395340 1115 ts_tablet_manager.cc:1939] T f6de0567e9894f54a4c5ff906c6fe7dd P 6549b657bcde4e29b2c2b20a833ed42b: Deleting consensus metadata
I20260811 03:16:25.395701 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.358536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35764 (local address 127.11.60.1:44313)
0811 03:16:25.358789 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.358798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.375102 (+ 16304us) server_negotiation.cc:407] Connection header received
0811 03:16:25.375599 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.375609 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.375702 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.375775 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:25.377740 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.379013 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.381029 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.381754 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.381883 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.387788 (+ 5905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:25.387806 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:25.387824 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:25.387879 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:25.391979 (+ 4100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:25.392964 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:25.392973 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:25.392977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:25.393145 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:25.393645 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:25.393650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:25.393654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:25.394195 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:25.394545 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.394986 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.395259 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 03:16:25.397281 543 catalog_manager.cc:4841] TS 6549b657bcde4e29b2c2b20a833ed42b (127.11.60.3:41559): tablet f6de0567e9894f54a4c5ff906c6fe7dd (table test-table [id=bea25422c2034bd8a06a5f095c30f0fa]) successfully deleted
I20260811 03:16:25.397480 759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:25.488546 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 625
I20260811 03:16:25.517539 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 774
Aug 11 03:16:25 dist-test-slave-tm1g krb5kdc[509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.2@KRBTEST.COM
I20260811 03:16:25.538043 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.518423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51904 (local address 127.11.60.2:42385)
0811 03:16:25.518643 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.518649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.518819 (+ 170us) server_negotiation.cc:407] Connection header received
0811 03:16:25.518989 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.518996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.519083 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.519204 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:25.520981 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.522295 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.524412 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.525042 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.525168 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.530909 (+ 5741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:25.530922 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:25.530939 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:25.530983 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:25.535138 (+ 4155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:25.536012 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:25.536021 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:25.536024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:25.536171 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:25.536562 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:25.536566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:25.536568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:25.536975 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:25.537209 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.537556 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.537729 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 03:16:25.538760 907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:25.631307 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 774
I20260811 03:16:25.652096 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 921
Aug 11 03:16:25 dist-test-slave-tm1g krb5kdc[509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.3@KRBTEST.COM
I20260811 03:16:25.671674 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:25.653040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40704 (local address 127.11.60.3:41559)
0811 03:16:25.653232 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:25.653237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:25.653348 (+ 111us) server_negotiation.cc:407] Connection header received
0811 03:16:25.653540 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:25.653545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:25.653607 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:25.653696 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:25.655649 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.656708 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.658739 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:25.659212 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:25.659313 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:25.665107 (+ 5794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:25.665124 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:25.665144 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:25.665195 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:25.668614 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:25.669399 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:25.669407 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:25.669409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:25.669593 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:25.670010 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:25.670014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:25.670016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:25.670457 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:25.670800 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:25.671063 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 03:16:25.671295 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 03:16:25.672384 1060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:25.803516 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 921
I20260811 03:16:25.827592 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 522
I20260811 03:16:25.828886 591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:25.942634 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 522
2026-08-11T03:16:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4682 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T03:16:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:26Z Disabled control of system clock
I20260811 03:16:26.037639 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:39911
--webserver_interface=127.11.60.62
--webserver_port=0
--builtin_ntp_servers=127.11.60.20:39775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:39911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:16:26.393415 1128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:26.393707 1128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:26.393790 1128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:26.400974 1128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:26.401079 1128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:26.401118 1128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:26.401173 1128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:26.401218 1128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:26.410238 1128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:39775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:39911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:26.411480 1128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:26.413247 1128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:26.425817 1133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:26.425729 1134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:26.425766 1138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:26.426403 1128 server_base.cc:1034] running on GCE node
I20260811 03:16:26.427491 1128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:26.428926 1128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:26.430171 1128 hybrid_clock.cc:648] HybridClock initialized: now 1786418186430132 us; error 56 us; skew 500 ppm
I20260811 03:16:26.430553 1128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:26.432821 1128 webserver.cc:460] Webserver started at http://127.11.60.62:39687/ using document root <none> and password file <none>
I20260811 03:16:26.433533 1128 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:26.433668 1128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:26.433995 1128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:26.436921 1128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "471c782c1d404dfa9490bd2a3e2a5d76"
format_stamp: "Formatted at 2026-08-11 03:16:26 on dist-test-slave-tm1g"
server_key: "076306e6f266149da9fcea779ccb5a32"
server_key_iv: "f37fb8985a1e66117bff9d33fb1869d1"
server_key_version: "encryptionkey@0"
I20260811 03:16:26.437825 1128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "471c782c1d404dfa9490bd2a3e2a5d76"
format_stamp: "Formatted at 2026-08-11 03:16:26 on dist-test-slave-tm1g"
server_key: "076306e6f266149da9fcea779ccb5a32"
server_key_iv: "f37fb8985a1e66117bff9d33fb1869d1"
server_key_version: "encryptionkey@0"
I20260811 03:16:26.443056 1128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 03:16:26.446563 1144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:26.447950 1128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:16:26.448155 1128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "471c782c1d404dfa9490bd2a3e2a5d76"
format_stamp: "Formatted at 2026-08-11 03:16:26 on dist-test-slave-tm1g"
server_key: "076306e6f266149da9fcea779ccb5a32"
server_key_iv: "f37fb8985a1e66117bff9d33fb1869d1"
server_key_version: "encryptionkey@0"
I20260811 03:16:26.448386 1128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:26.464607 1128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:26.465500 1128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:26.465792 1128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:26.485028 1128 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:39911
I20260811 03:16:26.485062 1202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:39911 every 8 connection(s)
I20260811 03:16:26.486474 1128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:26.488415 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 1128
I20260811 03:16:26.488596 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:26.489169 11504 external_mini_cluster.cc:1428] Setting key 2d492cccd84c3eb783d6c05db6e17018
I20260811 03:16:26.491355 1203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:26.503012 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:26.491210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33006 (local address 127.11.60.62:39911)
0811 03:16:26.492247 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:26.492258 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:26.492298 (+ 40us) server_negotiation.cc:407] Connection header received
0811 03:16:26.493957 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:26.493978 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:26.494489 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:26.494906 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:26.495469 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:26.496826 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:26.498220 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:26.498969 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:26.499178 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:26.500479 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:26.500486 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:26.500501 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:26.501142 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:26.501225 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:26.501247 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:26.501391 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:26.501645 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 03:16:26.501971 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":383,"threads_started":1}
I20260811 03:16:26.503960 1203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76: Bootstrap starting.
I20260811 03:16:26.507519 1203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:26.508533 1203 log.cc:824] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:26.511745 1203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76: No bootstrap required, opened a new log
I20260811 03:16:26.519124 1203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c782c1d404dfa9490bd2a3e2a5d76" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39911 } }
I20260811 03:16:26.519462 1203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:26.519541 1203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471c782c1d404dfa9490bd2a3e2a5d76, State: Initialized, Role: FOLLOWER
I20260811 03:16:26.520022 1203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c782c1d404dfa9490bd2a3e2a5d76" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39911 } }
I20260811 03:16:26.520223 1203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:26.520318 1203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:26.520450 1203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:26.522850 1203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c782c1d404dfa9490bd2a3e2a5d76" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39911 } }
I20260811 03:16:26.523311 1203 leader_election.cc:302] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 471c782c1d404dfa9490bd2a3e2a5d76; no voters:
I20260811 03:16:26.523877 1203 leader_election.cc:288] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:26.524102 1208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:26.524879 1208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [term 1 LEADER]: Becoming Leader. State: Replica: 471c782c1d404dfa9490bd2a3e2a5d76, State: Running, Role: LEADER
I20260811 03:16:26.525440 1208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c782c1d404dfa9490bd2a3e2a5d76" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39911 } }
I20260811 03:16:26.526091 1203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:26.528223 1209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "471c782c1d404dfa9490bd2a3e2a5d76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c782c1d404dfa9490bd2a3e2a5d76" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39911 } } }
I20260811 03:16:26.528599 1209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:26.529798 1214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:26.529348 1210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 471c782c1d404dfa9490bd2a3e2a5d76. Latest consensus state: current_term: 1 leader_uuid: "471c782c1d404dfa9490bd2a3e2a5d76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c782c1d404dfa9490bd2a3e2a5d76" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39911 } } }
I20260811 03:16:26.534901 1210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:26.543392 1214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:26.552857 1214 catalog_manager.cc:1347] Generated new cluster ID: 6ee9a5e8e9ac49f883d235cfecf64b5f
I20260811 03:16:26.552990 1214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:26.574970 1214 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:26.576351 1214 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:26.587472 1214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76: Generated new TSK 0
I20260811 03:16:26.588625 1214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:26.603169 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.1:0
--local_ip_for_outbound_sockets=127.11.60.1
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39911
--builtin_ntp_servers=127.11.60.20:39775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:16:26.965154 1229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:26.965457 1229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:26.965593 1229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:26.972755 1229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:26.972852 1229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:26.973011 1229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.1
I20260811 03:16:26.982074 1229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:39775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.60.1
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.1
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:26.983457 1229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:26.985371 1229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:26.998507 1235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:26.998471 1239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:26.998435 1236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:26.999240 1229 server_base.cc:1034] running on GCE node
I20260811 03:16:26.999820 1229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:27.000726 1229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:27.001968 1229 hybrid_clock.cc:648] HybridClock initialized: now 1786418187001904 us; error 56 us; skew 500 ppm
I20260811 03:16:27.002395 1229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:27.004948 1229 webserver.cc:460] Webserver started at http://127.11.60.1:45497/ using document root <none> and password file <none>
I20260811 03:16:27.005704 1229 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:27.005823 1229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:27.006163 1229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:27.009157 1229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/data/instance:
uuid: "06e33911af9b4a5181817b339b709d3f"
format_stamp: "Formatted at 2026-08-11 03:16:27 on dist-test-slave-tm1g"
server_key: "ee6048c8ad75b06a68882bf08371e1a1"
server_key_iv: "8db4bdf36c4788bcbe1cd7de500d41db"
server_key_version: "encryptionkey@0"
I20260811 03:16:27.009989 1229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance:
uuid: "06e33911af9b4a5181817b339b709d3f"
format_stamp: "Formatted at 2026-08-11 03:16:27 on dist-test-slave-tm1g"
server_key: "ee6048c8ad75b06a68882bf08371e1a1"
server_key_iv: "8db4bdf36c4788bcbe1cd7de500d41db"
server_key_version: "encryptionkey@0"
I20260811 03:16:27.015313 1229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:16:27.019017 1247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:27.020634 1229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:16:27.020834 1229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/wal
uuid: "06e33911af9b4a5181817b339b709d3f"
format_stamp: "Formatted at 2026-08-11 03:16:27 on dist-test-slave-tm1g"
server_key: "ee6048c8ad75b06a68882bf08371e1a1"
server_key_iv: "8db4bdf36c4788bcbe1cd7de500d41db"
server_key_version: "encryptionkey@0"
I20260811 03:16:27.021072 1229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:27.039511 1229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:27.040431 1229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:27.040735 1229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:27.041850 1229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:27.043452 1229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:27.043591 1229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:27.043745 1229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:27.043836 1229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:27.077999 1229 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.1:38351
I20260811 03:16:27.078033 1373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.1:38351 every 8 connection(s)
I20260811 03:16:27.079444 1229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/data/info.pb
I20260811 03:16:27.086486 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 1229
I20260811 03:16:27.086673 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-0/wal/instance
I20260811 03:16:27.087307 11504 external_mini_cluster.cc:1428] Setting key c44a62e2875f9a4042a201daa95bcb8b
I20260811 03:16:27.090339 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.2:0
--local_ip_for_outbound_sockets=127.11.60.2
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39911
--builtin_ntp_servers=127.11.60.20:39775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:16:27.093501 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:27.082205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.1:41849 (local address 127.11.60.62:39911)
0811 03:16:27.082705 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:27.082710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:27.083959 (+ 1249us) server_negotiation.cc:407] Connection header received
0811 03:16:27.085851 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:27.085856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:27.085951 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:27.086138 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:27.087647 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:27.088524 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:27.090376 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:27.090885 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:27.091038 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:27.092578 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:27.092583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:27.092586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:27.092607 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:27.092653 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:27.092667 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:27.092770 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:27.093012 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 03:16:27.093126 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":276,"threads_started":1}
I20260811 03:16:27.094666 1378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:27.082865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39911 (local address 127.11.60.1:41849)
0811 03:16:27.083787 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:27.083830 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:27.085548 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:27.086474 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:27.086483 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:27.086607 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:27.087335 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:27.087386 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:27.088692 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:27.088698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:27.090198 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:27.090212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:27.091145 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:27.091152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:27.091396 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:27.092183 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:16:27.092186 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:27.092230 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:16:27.092325 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:27.092778 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:27.092792 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:27.092802 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 03:16:27.093002 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 03:16:27.093451 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":277,"threads_started":1}
I20260811 03:16:27.096423 1374 heartbeater.cc:347] Connected to a master server at 127.11.60.62:39911
I20260811 03:16:27.096753 1374 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:27.097558 1374 heartbeater.cc:511] Master 127.11.60.62:39911 requested a full tablet report, sending...
I20260811 03:16:27.100093 1164 ts_manager.cc:194] Registered new tserver with Master: 06e33911af9b4a5181817b339b709d3f (127.11.60.1:38351)
I20260811 03:16:27.102349 1164 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.1:41849
W20260811 03:16:27.441471 1379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:27.441805 1379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:27.441962 1379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:27.449360 1379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:27.449515 1379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:27.449702 1379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.2
I20260811 03:16:27.459815 1379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:39775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.60.2
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.2
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:27.461236 1379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:27.463199 1379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:27.476879 1385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:27.476794 1387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:27.476835 1384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:27.477645 1379 server_base.cc:1034] running on GCE node
I20260811 03:16:27.478377 1379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:27.479354 1379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:27.480566 1379 hybrid_clock.cc:648] HybridClock initialized: now 1786418187480521 us; error 32 us; skew 500 ppm
I20260811 03:16:27.480899 1379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:27.483458 1379 webserver.cc:460] Webserver started at http://127.11.60.2:37533/ using document root <none> and password file <none>
I20260811 03:16:27.484236 1379 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:27.484326 1379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:27.484663 1379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:27.487643 1379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/data/instance:
uuid: "792d535813d4468884b8e0afa8d4c165"
format_stamp: "Formatted at 2026-08-11 03:16:27 on dist-test-slave-tm1g"
server_key: "1f8a6136fcd8bfd883a73b2e9103fc79"
server_key_iv: "60bb2ea48c306716579e392b3c2f2a20"
server_key_version: "encryptionkey@0"
I20260811 03:16:27.488462 1379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance:
uuid: "792d535813d4468884b8e0afa8d4c165"
format_stamp: "Formatted at 2026-08-11 03:16:27 on dist-test-slave-tm1g"
server_key: "1f8a6136fcd8bfd883a73b2e9103fc79"
server_key_iv: "60bb2ea48c306716579e392b3c2f2a20"
server_key_version: "encryptionkey@0"
I20260811 03:16:27.494004 1379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:16:27.497639 1393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:27.499094 1379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:16:27.499284 1379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/wal
uuid: "792d535813d4468884b8e0afa8d4c165"
format_stamp: "Formatted at 2026-08-11 03:16:27 on dist-test-slave-tm1g"
server_key: "1f8a6136fcd8bfd883a73b2e9103fc79"
server_key_iv: "60bb2ea48c306716579e392b3c2f2a20"
server_key_version: "encryptionkey@0"
I20260811 03:16:27.499542 1379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:27.517323 1379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:27.518210 1379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:27.518546 1379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:27.519603 1379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:27.521162 1379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:27.521293 1379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:27.521433 1379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:27.521534 1379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:27.557379 1379 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.2:39895
I20260811 03:16:27.557444 1519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.2:39895 every 8 connection(s)
I20260811 03:16:27.558785 1379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/data/info.pb
I20260811 03:16:27.565490 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 1379
I20260811 03:16:27.565680 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/wal/instance
I20260811 03:16:27.566273 11504 external_mini_cluster.cc:1428] Setting key 35a04b1cd6f295f2a98d1104bb29d653
I20260811 03:16:27.569175 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.60.3:0
--local_ip_for_outbound_sockets=127.11.60.3
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39911
--builtin_ntp_servers=127.11.60.20:39775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:16:27.572361 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:27.561460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:52769 (local address 127.11.60.62:39911)
0811 03:16:27.561702 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:27.561706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:27.563201 (+ 1495us) server_negotiation.cc:407] Connection header received
0811 03:16:27.564980 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:27.564985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:27.565058 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:27.565150 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:27.566699 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:27.567613 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:27.569482 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:27.569888 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:27.570027 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:27.571520 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:27.571525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:27.571529 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:27.571547 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:27.571581 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:27.571593 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:27.571692 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:27.571931 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 03:16:27.572036 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 03:16:27.573412 1522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:27.562005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39911 (local address 127.11.60.2:52769)
0811 03:16:27.563032 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:27.563083 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:27.564711 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:27.565495 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:27.565507 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:27.565630 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:27.566515 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:27.566542 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:27.567784 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:27.567790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:27.569308 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:27.569329 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:27.570141 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:27.570151 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:27.570439 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:27.571176 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:16:27.571179 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:27.571220 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:16:27.571301 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:27.571692 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:27.571706 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:27.571716 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 03:16:27.571902 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 03:16:27.572329 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":401,"threads_started":1}
I20260811 03:16:27.574843 1520 heartbeater.cc:347] Connected to a master server at 127.11.60.62:39911
I20260811 03:16:27.575187 1520 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:27.576138 1520 heartbeater.cc:511] Master 127.11.60.62:39911 requested a full tablet report, sending...
I20260811 03:16:27.577840 1164 ts_manager.cc:194] Registered new tserver with Master: 792d535813d4468884b8e0afa8d4c165 (127.11.60.2:39895)
I20260811 03:16:27.579094 1164 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.2:52769
W20260811 03:16:27.930831 1523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:27.931190 1523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:27.931376 1523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:27.938818 1523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:27.938980 1523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:27.939167 1523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.60.3
I20260811 03:16:27.949193 1523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:39775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.60.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.60.3
--webserver_port=0
--tserver_master_addrs=127.11.60.62:39911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.60.3
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:27.950711 1523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:27.952782 1523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:27.967185 1529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:27.967237 1530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:27.967213 1532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:27.968729 1523 server_base.cc:1034] running on GCE node
I20260811 03:16:27.969365 1523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:27.970418 1523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:27.971660 1523 hybrid_clock.cc:648] HybridClock initialized: now 1786418187971607 us; error 64 us; skew 500 ppm
I20260811 03:16:27.972034 1523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 03:16:27.974359 1523 webserver.cc:460] Webserver started at http://127.11.60.3:33173/ using document root <none> and password file <none>
I20260811 03:16:27.975108 1523 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:27.975263 1523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:27.975600 1523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:27.978582 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/data/instance:
uuid: "ca04476084104b2fadc3d75983e5298d"
format_stamp: "Formatted at 2026-08-11 03:16:27 on dist-test-slave-tm1g"
server_key: "00b674c0147a2d59bc3ae7ebf4907668"
server_key_iv: "65ebd391a8d249778851d58409b5cd05"
server_key_version: "encryptionkey@0"
I20260811 03:16:27.979400 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance:
uuid: "ca04476084104b2fadc3d75983e5298d"
format_stamp: "Formatted at 2026-08-11 03:16:27 on dist-test-slave-tm1g"
server_key: "00b674c0147a2d59bc3ae7ebf4907668"
server_key_iv: "65ebd391a8d249778851d58409b5cd05"
server_key_version: "encryptionkey@0"
I20260811 03:16:27.984547 1523 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260811 03:16:27.988035 1538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:27.989390 1523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:16:27.989573 1523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/wal
uuid: "ca04476084104b2fadc3d75983e5298d"
format_stamp: "Formatted at 2026-08-11 03:16:27 on dist-test-slave-tm1g"
server_key: "00b674c0147a2d59bc3ae7ebf4907668"
server_key_iv: "65ebd391a8d249778851d58409b5cd05"
server_key_version: "encryptionkey@0"
I20260811 03:16:27.989806 1523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:28.002233 1523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:28.003198 1523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:28.003500 1523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:28.004674 1523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:16:28.006783 1523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:16:28.006908 1523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:28.007059 1523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:16:28.007150 1523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:28.043197 1523 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.3:38451
I20260811 03:16:28.043292 1665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.3:38451 every 8 connection(s)
I20260811 03:16:28.044574 1523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/data/info.pb
I20260811 03:16:28.053642 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 1523
I20260811 03:16:28.053856 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/wal/instance
I20260811 03:16:28.054459 11504 external_mini_cluster.cc:1428] Setting key 2a9c5eea3e5007739610cdc1deba5c42
I20260811 03:16:28.059613 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.047787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.3:50309 (local address 127.11.60.62:39911)
0811 03:16:28.047986 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.047990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.049399 (+ 1409us) server_negotiation.cc:407] Connection header received
0811 03:16:28.051363 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.051368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.051444 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.051550 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:28.052911 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.054088 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.056397 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.057063 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.057209 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.058846 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:28.058851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:28.058855 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:28.058873 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:28.058907 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:28.058919 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:28.059010 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.059259 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.059349 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 03:16:28.060598 1670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.048308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39911 (local address 127.11.60.3:50309)
0811 03:16:28.049257 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.049294 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:28.051080 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:28.051876 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:28.051886 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:28.051999 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:16:28.052686 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.052704 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.054257 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.054267 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.056235 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.056253 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.057285 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.057290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.057650 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.058483 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:16:28.058486 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:16:28.058547 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:16:28.058629 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:16:28.059012 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:16:28.059032 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:16:28.059038 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 03:16:28.059180 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 03:16:28.059481 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":305,"threads_started":1}
I20260811 03:16:28.062088 1666 heartbeater.cc:347] Connected to a master server at 127.11.60.62:39911
I20260811 03:16:28.062456 1666 heartbeater.cc:464] Registering TS with master...
I20260811 03:16:28.063213 1666 heartbeater.cc:511] Master 127.11.60.62:39911 requested a full tablet report, sending...
I20260811 03:16:28.064628 1164 ts_manager.cc:194] Registered new tserver with Master: ca04476084104b2fadc3d75983e5298d (127.11.60.3:38451)
I20260811 03:16:28.065464 1164 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.60.3:50309
I20260811 03:16:28.070461 11504 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:16:28.081851 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.074970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.11.60.62:39911)
0811 03:16:28.075225 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.075232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.075595 (+ 363us) server_negotiation.cc:407] Connection header received
0811 03:16:28.075733 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.075737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.075812 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.075897 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:28.076412 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.077145 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.079310 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.079736 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.079836 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.081064 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:28.081069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:28.081072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:28.081091 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:28.081123 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:28.081136 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:28.081206 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.081371 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.081481 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 03:16:28.085907 1164 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:16:28.092969 1164 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:16:28.105873 1374 heartbeater.cc:503] Master 127.11.60.62:39911 was elected leader, sending a full tablet report...
I20260811 03:16:28.117916 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.106995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38664 (local address 127.11.60.2:39895)
0811 03:16:28.108282 (+ 1287us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.108292 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.109351 (+ 1059us) server_negotiation.cc:407] Connection header received
0811 03:16:28.109714 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.109722 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.110007 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.110191 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:28.111247 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.112898 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.115612 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.116689 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.116854 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.117118 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.117374 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.117551 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1115,"thread_start_us":1019,"threads_started":1}
I20260811 03:16:28.117974 1683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.106779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.2:39895 (local address 127.0.0.1:38664)
0811 03:16:28.109168 (+ 2389us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.109215 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:28.109410 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:28.110365 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:28.110372 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:28.110403 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:28.110999 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.111024 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.113065 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.113074 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.115382 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.115397 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.116901 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.116907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.117111 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.117172 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 03:16:28.117318 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 03:16:28.117524 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2165,"mutex_wait_us":238,"thread_start_us":1983,"threads_started":1}
I20260811 03:16:28.119192 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.106132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:38351 (local address 127.0.0.1:39338)
0811 03:16:28.109468 (+ 3336us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.109487 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:28.109615 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:28.110617 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:28.110621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:28.110642 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:28.111157 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.111173 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.113084 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.113092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.115693 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.115715 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.118235 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.118241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.118581 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.118605 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:16:28.118710 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 03:16:28.118855 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2751,"mutex_wait_us":227,"thread_start_us":2403,"threads_started":1}
I20260811 03:16:28.119343 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.106670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39338 (local address 127.11.60.1:38351)
0811 03:16:28.107632 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.107645 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.109785 (+ 2140us) server_negotiation.cc:407] Connection header received
0811 03:16:28.109889 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.109897 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.110143 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.110348 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:28.111398 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.112883 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.115925 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.117664 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.117797 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.118094 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.118803 (+ 709us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.118983 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":991,"threads_started":1}
I20260811 03:16:28.121503 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.109428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:38451 (local address 127.0.0.1:52720)
0811 03:16:28.111788 (+ 2360us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.111809 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:28.111937 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:28.112498 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:28.112503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:28.112528 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:28.113132 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.113143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.115210 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.115216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.118571 (+ 3355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.118589 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.120743 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.120749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.120911 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.120955 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 03:16:28.121052 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 03:16:28.121180 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2241,"mutex_wait_us":2,"thread_start_us":2095,"threads_started":1}
I20260811 03:16:28.121683 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.110085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52720 (local address 127.11.60.3:38451)
0811 03:16:28.111365 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.111373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.111936 (+ 563us) server_negotiation.cc:407] Connection header received
0811 03:16:28.112072 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.112080 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.112284 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.112484 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:28.113400 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.115054 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.118819 (+ 3765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.120207 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.120346 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.120619 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.121136 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.121301 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1093,"thread_start_us":926,"threads_started":1}
I20260811 03:16:28.122516 1446 tablet_service.cc:1467] Processing CreateTablet for tablet 19504bf918ca477c9c004e49cdc891bb (DEFAULT_TABLE table=test-table [id=92435ab229d94b2780963407a63c5c1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:28.122825 1302 tablet_service.cc:1467] Processing CreateTablet for tablet 19504bf918ca477c9c004e49cdc891bb (DEFAULT_TABLE table=test-table [id=92435ab229d94b2780963407a63c5c1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:28.123852 1446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19504bf918ca477c9c004e49cdc891bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:28.124241 1302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19504bf918ca477c9c004e49cdc891bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:28.125921 1595 tablet_service.cc:1467] Processing CreateTablet for tablet 19504bf918ca477c9c004e49cdc891bb (DEFAULT_TABLE table=test-table [id=92435ab229d94b2780963407a63c5c1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:16:28.127475 1595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19504bf918ca477c9c004e49cdc891bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:28.135766 1687 tablet_bootstrap.cc:492] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: Bootstrap starting.
I20260811 03:16:28.139437 1687 tablet_bootstrap.cc:654] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:28.140177 1688 tablet_bootstrap.cc:492] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f: Bootstrap starting.
I20260811 03:16:28.140966 1687 log.cc:824] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:28.144220 1688 tablet_bootstrap.cc:654] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:28.144467 1687 tablet_bootstrap.cc:492] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: No bootstrap required, opened a new log
I20260811 03:16:28.144768 1687 ts_tablet_manager.cc:1397] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 03:16:28.145752 1688 log.cc:824] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:28.146618 1690 tablet_bootstrap.cc:492] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: Bootstrap starting.
I20260811 03:16:28.149037 1688 tablet_bootstrap.cc:492] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f: No bootstrap required, opened a new log
I20260811 03:16:28.149315 1688 ts_tablet_manager.cc:1397] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260811 03:16:28.149567 1690 tablet_bootstrap.cc:654] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:28.150636 1690 log.cc:824] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:28.153754 1690 tablet_bootstrap.cc:492] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: No bootstrap required, opened a new log
I20260811 03:16:28.154089 1690 ts_tablet_manager.cc:1397] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260811 03:16:28.155460 1687 raft_consensus.cc:348] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 } } peers { permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 } } peers { permanent_uuid: "792d535813d4468884b8e0afa8d4c165" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 39895 } }
I20260811 03:16:28.155958 1687 raft_consensus.cc:374] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:28.156067 1687 raft_consensus.cc:729] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792d535813d4468884b8e0afa8d4c165, State: Initialized, Role: FOLLOWER
I20260811 03:16:28.156754 1687 consensus_queue.cc:260] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 } } peers { permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 } } peers { permanent_uuid: "792d535813d4468884b8e0afa8d4c165" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 39895 } }
I20260811 03:16:28.158126 1520 heartbeater.cc:503] Master 127.11.60.62:39911 was elected leader, sending a full tablet report...
I20260811 03:16:28.158788 1687 ts_tablet_manager.cc:1428] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260811 03:16:28.161119 1688 raft_consensus.cc:348] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 } } peers { permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 } } peers { permanent_uuid: "792d535813d4468884b8e0afa8d4c165" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 39895 } }
I20260811 03:16:28.161507 1688 raft_consensus.cc:374] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:28.161638 1688 raft_consensus.cc:729] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e33911af9b4a5181817b339b709d3f, State: Initialized, Role: FOLLOWER
I20260811 03:16:28.162250 1688 consensus_queue.cc:260] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 } } peers { permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 } } peers { permanent_uuid: "792d535813d4468884b8e0afa8d4c165" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 39895 } }
I20260811 03:16:28.163326 1690 raft_consensus.cc:348] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 } } peers { permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 } } peers { permanent_uuid: "792d535813d4468884b8e0afa8d4c165" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 39895 } }
I20260811 03:16:28.163877 1690 raft_consensus.cc:374] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:28.164024 1690 raft_consensus.cc:729] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca04476084104b2fadc3d75983e5298d, State: Initialized, Role: FOLLOWER
I20260811 03:16:28.164337 1688 ts_tablet_manager.cc:1428] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260811 03:16:28.164814 1690 consensus_queue.cc:260] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 } } peers { permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 } } peers { permanent_uuid: "792d535813d4468884b8e0afa8d4c165" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 39895 } }
I20260811 03:16:28.166208 1666 heartbeater.cc:503] Master 127.11.60.62:39911 was elected leader, sending a full tablet report...
I20260811 03:16:28.167485 1690 ts_tablet_manager.cc:1428] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
W20260811 03:16:28.296485 1668 tablet.cc:2369] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:16:28.311014 1521 tablet.cc:2369] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:16:28.332295 1375 tablet.cc:2369] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:16:28.466863 1693 raft_consensus.cc:482] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:16:28.467283 1693 raft_consensus.cc:504] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 } } peers { permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 } } peers { permanent_uuid: "792d535813d4468884b8e0afa8d4c165" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 39895 } }
I20260811 03:16:28.469182 1693 leader_election.cc:288] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06e33911af9b4a5181817b339b709d3f (127.11.60.1:38351), ca04476084104b2fadc3d75983e5298d (127.11.60.3:38451)
I20260811 03:16:28.477082 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.469830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:55733 (local address 127.11.60.1:38351)
0811 03:16:28.470104 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.470113 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.470376 (+ 263us) server_negotiation.cc:407] Connection header received
0811 03:16:28.470763 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.470771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.470899 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.471019 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:28.471747 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.472806 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.475036 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.476129 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.476227 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.476306 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.476691 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.476808 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 03:16:28.477190 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.469468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:38351 (local address 127.11.60.2:55733)
0811 03:16:28.470230 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.470247 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:28.470545 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:28.471060 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:28.471065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:28.471090 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:28.471592 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.471607 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.472999 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.473005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.474818 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.474833 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.476327 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.476334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.476517 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.476557 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 03:16:28.476662 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 03:16:28.476798 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":1,"thread_start_us":352,"threads_started":1}
I20260811 03:16:28.477926 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.470471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.60.2:50473 (local address 127.11.60.3:38451)
0811 03:16:28.470697 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.470703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.471438 (+ 735us) server_negotiation.cc:407] Connection header received
0811 03:16:28.471533 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.471538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.471612 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.471710 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:16:28.472408 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.473518 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.475719 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.476730 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.476844 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.476938 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.477493 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.477676 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 03:16:28.478575 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.470541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.3:38451 (local address 127.11.60.2:50473)
0811 03:16:28.471243 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.471265 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 03:16:28.471412 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:16:28.471827 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:16:28.471832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:16:28.471861 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:16:28.472230 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.472241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.473682 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.473686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.475527 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:16:28.475543 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.476937 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.476944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:16:28.477138 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.477178 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 03:16:28.477938 (+ 760us) client_negotiation.cc:240] Negotiation successful
0811 03:16:28.478125 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":21,"thread_start_us":321,"threads_started":1}
I20260811 03:16:28.478530 1324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19504bf918ca477c9c004e49cdc891bb" candidate_uuid: "792d535813d4468884b8e0afa8d4c165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e33911af9b4a5181817b339b709d3f" is_pre_election: true
I20260811 03:16:28.479172 1324 raft_consensus.cc:2454] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792d535813d4468884b8e0afa8d4c165 in term 0.
I20260811 03:16:28.479678 1617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19504bf918ca477c9c004e49cdc891bb" candidate_uuid: "792d535813d4468884b8e0afa8d4c165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca04476084104b2fadc3d75983e5298d" is_pre_election: true
I20260811 03:16:28.479939 1394 leader_election.cc:302] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e33911af9b4a5181817b339b709d3f, 792d535813d4468884b8e0afa8d4c165; no voters:
I20260811 03:16:28.480230 1617 raft_consensus.cc:2454] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792d535813d4468884b8e0afa8d4c165 in term 0.
I20260811 03:16:28.480378 1693 raft_consensus.cc:2781] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:16:28.480602 1693 raft_consensus.cc:482] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:16:28.480724 1693 raft_consensus.cc:3037] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:28.483773 1693 raft_consensus.cc:504] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 } } peers { permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 } } peers { permanent_uuid: "792d535813d4468884b8e0afa8d4c165" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 39895 } }
I20260811 03:16:28.484472 1693 leader_election.cc:288] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [CANDIDATE]: Term 1 election: Requested vote from peers 06e33911af9b4a5181817b339b709d3f (127.11.60.1:38351), ca04476084104b2fadc3d75983e5298d (127.11.60.3:38451)
I20260811 03:16:28.485106 1324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19504bf918ca477c9c004e49cdc891bb" candidate_uuid: "792d535813d4468884b8e0afa8d4c165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e33911af9b4a5181817b339b709d3f"
I20260811 03:16:28.485379 1324 raft_consensus.cc:3037] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:28.485385 1617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19504bf918ca477c9c004e49cdc891bb" candidate_uuid: "792d535813d4468884b8e0afa8d4c165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca04476084104b2fadc3d75983e5298d"
I20260811 03:16:28.485602 1617 raft_consensus.cc:3037] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:28.488879 1617 raft_consensus.cc:2454] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792d535813d4468884b8e0afa8d4c165 in term 1.
I20260811 03:16:28.488884 1324 raft_consensus.cc:2454] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792d535813d4468884b8e0afa8d4c165 in term 1.
I20260811 03:16:28.489337 1394 leader_election.cc:302] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792d535813d4468884b8e0afa8d4c165, ca04476084104b2fadc3d75983e5298d; no voters:
I20260811 03:16:28.489667 1693 raft_consensus.cc:2781] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:28.490232 1693 raft_consensus.cc:686] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 1 LEADER]: Becoming Leader. State: Replica: 792d535813d4468884b8e0afa8d4c165, State: Running, Role: LEADER
I20260811 03:16:28.491261 1693 consensus_queue.cc:237] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 } } peers { permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 } } peers { permanent_uuid: "792d535813d4468884b8e0afa8d4c165" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 39895 } }
I20260811 03:16:28.496692 1164 catalog_manager.cc:5495] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792d535813d4468884b8e0afa8d4c165 (127.11.60.2). New cstate: current_term: 1 leader_uuid: "792d535813d4468884b8e0afa8d4c165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792d535813d4468884b8e0afa8d4c165" member_type: VOTER last_known_addr { host: "127.11.60.2" port: 39895 } health_report { overall_health: HEALTHY } } }
I20260811 03:16:28.582926 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.577523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48946 (local address 127.11.60.2:39895)
0811 03:16:28.577721 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.577725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.577744 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:16:28.577832 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.577836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.577930 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.578062 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:16:28.578426 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.579130 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.580637 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.581093 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.581201 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.581686 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:16:28.582231 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:16:28.582491 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.582538 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.582611 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 03:16:28.591775 1324 raft_consensus.cc:1261] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f [term 1 FOLLOWER]: Refusing update from remote peer 792d535813d4468884b8e0afa8d4c165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:28.591775 1617 raft_consensus.cc:1261] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d [term 1 FOLLOWER]: Refusing update from remote peer 792d535813d4468884b8e0afa8d4c165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:16:28.592832 1698 consensus_queue.cc:1035] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06e33911af9b4a5181817b339b709d3f" member_type: VOTER last_known_addr { host: "127.11.60.1" port: 38351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:28.593263 1693 consensus_queue.cc:1035] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca04476084104b2fadc3d75983e5298d" member_type: VOTER last_known_addr { host: "127.11.60.3" port: 38451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:16:28.604642 1702 mvcc.cc:205] Tried to move back new op lower bound from 7317168900460072960 to 7317168900064653312. Current Snapshot: MvccSnapshot[applied={T|T < 7317168900460072960}]
I20260811 03:16:28.606119 1704 mvcc.cc:205] Tried to move back new op lower bound from 7317168900460072960 to 7317168900064653312. Current Snapshot: MvccSnapshot[applied={T|T < 7317168900460072960}]
I20260811 03:16:28.618726 1163 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:28.619010 1163 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:16:28.622681 1163 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 471c782c1d404dfa9490bd2a3e2a5d76: Sending DeleteTablet for 3 replicas of tablet 19504bf918ca477c9c004e49cdc891bb
I20260811 03:16:28.623924 1595 tablet_service.cc:1514] Processing DeleteTablet for tablet 19504bf918ca477c9c004e49cdc891bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:28 UTC) from {username='slave'} at 127.0.0.1:52720
I20260811 03:16:28.623854 1302 tablet_service.cc:1514] Processing DeleteTablet for tablet 19504bf918ca477c9c004e49cdc891bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:28 UTC) from {username='slave'} at 127.0.0.1:39338
I20260811 03:16:28.624135 1446 tablet_service.cc:1514] Processing DeleteTablet for tablet 19504bf918ca477c9c004e49cdc891bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:16:28 UTC) from {username='slave'} at 127.0.0.1:38664
I20260811 03:16:28.626928 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 1229
I20260811 03:16:28.635275 1712 tablet_replica.cc:330] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: stopping tablet replica
I20260811 03:16:28.636313 1714 tablet_replica.cc:330] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f: stopping tablet replica
I20260811 03:16:28.636672 1712 raft_consensus.cc:2229] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:28.636801 1714 raft_consensus.cc:2229] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:16:28.637264 1712 raft_consensus.cc:2258] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:28.637422 1714 raft_consensus.cc:2258] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:28.640136 1714 ts_tablet_manager.cc:1905] T 19504bf918ca477c9c004e49cdc891bb P 06e33911af9b4a5181817b339b709d3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:28.641302 1715 tablet_replica.cc:330] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: stopping tablet replica
I20260811 03:16:28.641737 1715 raft_consensus.cc:2229] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:16:28.642493 1715 raft_consensus.cc:2258] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:16:28.643908 1712 ts_tablet_manager.cc:1905] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:28.645357 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.628884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36018 (local address 127.11.60.1:38351)
0811 03:16:28.629130 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.629141 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.635626 (+ 6485us) server_negotiation.cc:407] Connection header received
0811 03:16:28.635848 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.635860 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.636042 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.636165 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:28.636946 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.638331 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.640725 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.641374 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.641531 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.643225 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:28.643235 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:28.643251 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:28.644021 (+ 770us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:28.644084 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:28.644112 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:28.644210 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.644819 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.644950 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 03:16:28.646373 1369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:28.648238 1715 ts_tablet_manager.cc:1905] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:16:28.652835 1712 ts_tablet_manager.cc:1918] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:28.653151 1712 log.cc:1196] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-2/wal/wals/19504bf918ca477c9c004e49cdc891bb
I20260811 03:16:28.653803 1712 ts_tablet_manager.cc:1939] T 19504bf918ca477c9c004e49cdc891bb P ca04476084104b2fadc3d75983e5298d: Deleting consensus metadata
I20260811 03:16:28.655966 1715 ts_tablet_manager.cc:1918] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:16:28.656163 1715 log.cc:1196] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: Deleting WAL directory at /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417858274425-11504-0/minicluster-data/ts-1/wal/wals/19504bf918ca477c9c004e49cdc891bb
I20260811 03:16:28.656389 1145 catalog_manager.cc:4841] TS ca04476084104b2fadc3d75983e5298d (127.11.60.3:38451): tablet 19504bf918ca477c9c004e49cdc891bb (table test-table [id=92435ab229d94b2780963407a63c5c1e]) successfully deleted
I20260811 03:16:28.656872 1715 ts_tablet_manager.cc:1939] T 19504bf918ca477c9c004e49cdc891bb P 792d535813d4468884b8e0afa8d4c165: Deleting consensus metadata
I20260811 03:16:28.659056 1146 catalog_manager.cc:4841] TS 792d535813d4468884b8e0afa8d4c165 (127.11.60.2:39895): tablet 19504bf918ca477c9c004e49cdc891bb (table test-table [id=92435ab229d94b2780963407a63c5c1e]) successfully deleted
I20260811 03:16:28.748474 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 1229
W20260811 03:16:28.774044 1145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.60.1:38351 (error 108)
I20260811 03:16:28.774343 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 1379
I20260811 03:16:28.777040 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.775828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:38351 (local address 127.0.0.1:36022)
0811 03:16:28.776720 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.776857 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":635,"threads_started":1}
W20260811 03:16:28.777365 1145 catalog_manager.cc:4584] TS 06e33911af9b4a5181817b339b709d3f (127.11.60.1:38351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19504bf918ca477c9c004e49cdc891bb: Network error: Client connection negotiation failed: client connection to 127.11.60.1:38351: connect: Connection refused (error 111)
I20260811 03:16:28.783231 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.775479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48952 (local address 127.11.60.2:39895)
0811 03:16:28.775698 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.775705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.775732 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:16:28.775813 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.775819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.775969 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.776127 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:28.776626 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.777857 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.779497 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.780180 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.780324 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.781437 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:28.781449 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:28.781469 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:28.782366 (+ 897us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:28.782435 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:28.782461 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:28.782579 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.782712 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.782823 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 03:16:28.784140 1515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:28.831103 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.830590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:38351 (local address 127.0.0.1:36030)
0811 03:16:28.830828 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.830944 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 03:16:28.832260 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.831867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:38351 (local address 127.0.0.1:36034)
0811 03:16:28.832056 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.832126 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 03:16:28.893177 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.892586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:38351 (local address 127.0.0.1:36044)
0811 03:16:28.892853 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.892952 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 03:16:28.894506 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.894061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:38351 (local address 127.0.0.1:36060)
0811 03:16:28.894194 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.894346 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 03:16:28.900709 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 1379
I20260811 03:16:28.922225 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 1523
I20260811 03:16:28.933923 1686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.923170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35718 (local address 127.11.60.3:38451)
0811 03:16:28.923362 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:28.923369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:28.923479 (+ 110us) server_negotiation.cc:407] Connection header received
0811 03:16:28.923714 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:28.923719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:28.923814 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:28.923931 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:28.924385 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.925161 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.926756 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:28.927234 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:28.927372 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:28.932425 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:28.932436 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:28.932451 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:28.933069 (+ 618us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:28.933138 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:28.933159 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:28.933291 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:28.933454 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 03:16:28.933574 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 03:16:28.934880 1660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:28.975281 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.974717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:38351 (local address 127.0.0.1:36074)
0811 03:16:28.974975 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.975089 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 03:16:28.976516 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:28.976029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:38351 (local address 127.0.0.1:36084)
0811 03:16:28.976269 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:28.976359 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260811 03:16:29.026645 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 1523
I20260811 03:16:29.048691 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 1128
I20260811 03:16:29.050007 1197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:29.161978 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:29.161274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.1:38351 (local address 127.0.0.1:36086)
0811 03:16:29.161566 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 03:16:29.161745 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.1:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 03:16:29.162160 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 1128
2026-08-11T03:16:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3220 ms)
[----------] 6 tests from EncryptionPolicyTest (23173 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:29 dist-test-slave-tm1g krb5kdc[1726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:29 dist-test-slave-tm1g krb5kdc[1726](info): set up 2 sockets
Aug 11 03:16:29 dist-test-slave-tm1g krb5kdc[1726](info): commencing operation
krb5kdc: starting...
W20260811 03:16:30.365164 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.015s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:30 dist-test-slave-tm1g krb5kdc[1726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:30.698932 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32855
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:31.054524 1744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:31.054894 1744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:31.055079 1744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:31.062354 1744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:31.062516 1744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:31.062623 1744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:31.062739 1744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:31.062827 1744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:31.072929 1744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32855
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:31.074352 1744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:31.075727 1744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:31.088030 1750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:31.088125 1753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:31.087997 1751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:31.089241 1744 server_base.cc:1034] running on GCE node
I20260811 03:16:31.090111 1744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:31.091671 1744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:31.092949 1744 hybrid_clock.cc:648] HybridClock initialized: now 1786418191092880 us; error 59 us; skew 500 ppm
I20260811 03:16:31.093326 1744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:31 dist-test-slave-tm1g krb5kdc[1726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418191, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:31.096915 1744 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:31.098611 1744 webserver.cc:460] Webserver started at http://127.0.0.1:39941/ using document root <none> and password file <none>
I20260811 03:16:31.099371 1744 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:31.099534 1744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:31.099860 1744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:31.103173 1744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "159da46c9d3d486fb7ba9bdf152502fa"
format_stamp: "Formatted at 2026-08-11 03:16:31 on dist-test-slave-tm1g"
server_key: "50e8e3b86216a2ccf9192ea00a8ffd4a"
server_key_iv: "5473f768b23219e63af36601a6a25ad8"
server_key_version: "encryptionkey@0"
I20260811 03:16:31.104019 1744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "159da46c9d3d486fb7ba9bdf152502fa"
format_stamp: "Formatted at 2026-08-11 03:16:31 on dist-test-slave-tm1g"
server_key: "50e8e3b86216a2ccf9192ea00a8ffd4a"
server_key_iv: "5473f768b23219e63af36601a6a25ad8"
server_key_version: "encryptionkey@0"
I20260811 03:16:31.109508 1744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 03:16:31.113188 1762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:31.114728 1744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:16:31.114928 1744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "159da46c9d3d486fb7ba9bdf152502fa"
format_stamp: "Formatted at 2026-08-11 03:16:31 on dist-test-slave-tm1g"
server_key: "50e8e3b86216a2ccf9192ea00a8ffd4a"
server_key_iv: "5473f768b23219e63af36601a6a25ad8"
server_key_version: "encryptionkey@0"
I20260811 03:16:31.115137 1744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:31.137573 1744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:31.141866 1744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:31.142185 1744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:31.162123 1744 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32855
I20260811 03:16:31.162216 1815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32855 every 8 connection(s)
I20260811 03:16:31.163662 1744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:31.168311 1817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:31.171409 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 1744
I20260811 03:16:31.171669 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:31.172586 11504 external_mini_cluster.cc:1428] Setting key 7ac2c992483c88e6d333048a20a5d760
I20260811 03:16:31.181972 1817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa: Bootstrap starting.
I20260811 03:16:31.185433 1817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:31.186563 1817 log.cc:824] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa: Log is configured to *not* fsync() on all Append() calls
Aug 11 03:16:31 dist-test-slave-tm1g krb5kdc[1726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 03:16:31.189958 1817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa: No bootstrap required, opened a new log
I20260811 03:16:31.199380 1820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:31.174996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55225 (local address 127.0.0.1:32855)
0811 03:16:31.175910 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:31.175923 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:31.175965 (+ 42us) server_negotiation.cc:407] Connection header received
0811 03:16:31.177785 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:31.177815 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:31.178250 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:31.178770 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:31.180276 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:31.181768 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:31.183239 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:31.183966 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:31.184166 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:31.189073 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:31.189081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:31.189096 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:31.189138 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:31.192713 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:31.194284 (+ 1571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:31.194292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:31.194298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:31.195038 (+ 740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:31.195837 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:31.195842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:31.195844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:31.196641 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:31.196952 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:31.197656 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 03:16:31.198117 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":422,"threads_started":1}
I20260811 03:16:31.200754 1817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159da46c9d3d486fb7ba9bdf152502fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32855 } }
I20260811 03:16:31.201164 1817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:31.201313 1817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 159da46c9d3d486fb7ba9bdf152502fa, State: Initialized, Role: FOLLOWER
I20260811 03:16:31.201897 1817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159da46c9d3d486fb7ba9bdf152502fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32855 } }
I20260811 03:16:31.202167 1817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:31.202353 1817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:31.202569 1817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:31.203943 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:31.205207 1817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159da46c9d3d486fb7ba9bdf152502fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32855 } }
I20260811 03:16:31.205780 1817 leader_election.cc:302] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 159da46c9d3d486fb7ba9bdf152502fa; no voters:
I20260811 03:16:31.206480 1817 leader_election.cc:288] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:31.206629 1822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:31.210273 1822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [term 1 LEADER]: Becoming Leader. State: Replica: 159da46c9d3d486fb7ba9bdf152502fa, State: Running, Role: LEADER
I20260811 03:16:31.211190 1822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159da46c9d3d486fb7ba9bdf152502fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32855 } }
I20260811 03:16:31.211611 1817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:31.216198 1823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "159da46c9d3d486fb7ba9bdf152502fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159da46c9d3d486fb7ba9bdf152502fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32855 } } }
I20260811 03:16:31.216617 1823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:31.217232 1825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 159da46c9d3d486fb7ba9bdf152502fa. Latest consensus state: current_term: 1 leader_uuid: "159da46c9d3d486fb7ba9bdf152502fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159da46c9d3d486fb7ba9bdf152502fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32855 } } }
I20260811 03:16:31.217582 1825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:31.235659 1820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:31.210410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39913 (local address 127.0.0.1:32855)
0811 03:16:31.210573 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:31.210579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:31.211289 (+ 710us) server_negotiation.cc:407] Connection header received
0811 03:16:31.211547 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:31.211553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:31.211639 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:31.211720 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:31.214154 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:31.215902 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:31.218522 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:31.219748 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:31.219934 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:31.224458 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:31.224468 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:31.224476 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:31.224538 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:31.229584 (+ 5046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:31.231033 (+ 1449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:31.231042 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:31.231048 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:31.231239 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:31.231970 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:31.231978 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:31.231984 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:31.232658 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:31.233045 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:31.234836 (+ 1791us) server_negotiation.cc:299] Negotiation successful
0811 03:16:31.235114 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 03:16:31.237592 1835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:31.243484 1835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:31.253887 1835 catalog_manager.cc:1347] Generated new cluster ID: fc9289af699949f796f6047d6bcc687e
I20260811 03:16:31.254029 1835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:31.282904 1835 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:31.284461 1835 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:31.294138 1835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 159da46c9d3d486fb7ba9bdf152502fa: Generated new TSK 0
I20260811 03:16:31.295164 1835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:31.308622 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 1744
I20260811 03:16:31.310822 1810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:31.411669 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 1744
W20260811 03:16:31.433065 1724 connection.cc:535] client connection to 127.0.0.1:32855 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32855): Connection reset by peer (error 104)
2026-08-11T03:16:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2242 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:31 dist-test-slave-tm1g krb5kdc[1853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:31 dist-test-slave-tm1g krb5kdc[1853](info): set up 2 sockets
Aug 11 03:16:31 dist-test-slave-tm1g krb5kdc[1853](info): commencing operation
krb5kdc: starting...
W20260811 03:16:32.602613 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:32 dist-test-slave-tm1g krb5kdc[1853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:32.938136 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:33.290319 1868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:33.290642 1868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:33.290802 1868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:33.298197 1868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:33.298362 1868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:33.298499 1868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:33.298604 1868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:33.298681 1868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:33.308845 1868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38467
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:33.310246 1868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:33.311668 1868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:33.323980 1873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:33.324061 1876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:33.324029 1874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:33.324891 1868 server_base.cc:1034] running on GCE node
I20260811 03:16:33.325692 1868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:33.327211 1868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:33.328440 1868 hybrid_clock.cc:648] HybridClock initialized: now 1786418193328397 us; error 52 us; skew 500 ppm
I20260811 03:16:33.328784 1868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:33 dist-test-slave-tm1g krb5kdc[1853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418193, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:33.332023 1868 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:33.333447 1868 webserver.cc:460] Webserver started at http://127.0.0.1:39333/ using document root <none> and password file <none>
I20260811 03:16:33.334097 1868 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:33.334200 1868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:33.334535 1868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:33.337466 1868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "362c248afd3e467fb55079fb0b814ba2"
format_stamp: "Formatted at 2026-08-11 03:16:33 on dist-test-slave-tm1g"
server_key: "3d07a54274043a792eae05cb9eb7380d"
server_key_iv: "baeda89578e3aef1b39c54594973c69c"
server_key_version: "encryptionkey@0"
I20260811 03:16:33.338194 1868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "362c248afd3e467fb55079fb0b814ba2"
format_stamp: "Formatted at 2026-08-11 03:16:33 on dist-test-slave-tm1g"
server_key: "3d07a54274043a792eae05cb9eb7380d"
server_key_iv: "baeda89578e3aef1b39c54594973c69c"
server_key_version: "encryptionkey@0"
I20260811 03:16:33.343580 1868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:16:33.347141 1884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:33.348502 1868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:16:33.348644 1868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "362c248afd3e467fb55079fb0b814ba2"
format_stamp: "Formatted at 2026-08-11 03:16:33 on dist-test-slave-tm1g"
server_key: "3d07a54274043a792eae05cb9eb7380d"
server_key_iv: "baeda89578e3aef1b39c54594973c69c"
server_key_version: "encryptionkey@0"
I20260811 03:16:33.348815 1868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:33.377247 1868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:33.381886 1868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:33.382205 1868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:33.403153 1868 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38467
I20260811 03:16:33.403185 1942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38467 every 8 connection(s)
I20260811 03:16:33.404565 1868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:33.408625 1943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:33.410377 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 1868
I20260811 03:16:33.410585 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:33.411155 11504 external_mini_cluster.cc:1428] Setting key 172d8f685e2e105304842fe1b49d1227
I20260811 03:16:33.423813 1943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2: Bootstrap starting.
I20260811 03:16:33.425604 1946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:33.413156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59943 (local address 127.0.0.1:38467)
0811 03:16:33.413994 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:33.414007 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:33.414051 (+ 44us) server_negotiation.cc:407] Connection header received
0811 03:16:33.415742 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:33.415768 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:33.416189 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:33.416534 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:33.417167 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:33.418586 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:33.420753 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:33.421550 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:33.421695 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:33.423157 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:33.423165 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:33.423182 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:33.423764 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:33.423854 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:33.423884 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:33.424074 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:33.424372 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 03:16:33.424549 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":384,"threads_started":1}
I20260811 03:16:33.427390 1943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:33.428658 1943 log.cc:824] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:33.429617 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:33.431854 1943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2: No bootstrap required, opened a new log
I20260811 03:16:33.439754 1943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c248afd3e467fb55079fb0b814ba2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260811 03:16:33.440320 1943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:33.440435 1943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362c248afd3e467fb55079fb0b814ba2, State: Initialized, Role: FOLLOWER
I20260811 03:16:33.441114 1943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c248afd3e467fb55079fb0b814ba2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260811 03:16:33.441381 1943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:33.441535 1943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:33.441712 1943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:33.444635 1943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c248afd3e467fb55079fb0b814ba2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260811 03:16:33.445261 1943 leader_election.cc:302] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 362c248afd3e467fb55079fb0b814ba2; no voters:
I20260811 03:16:33.445497 1946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:33.434530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45701 (local address 127.0.0.1:38467)
0811 03:16:33.434721 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:33.434727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:33.438936 (+ 4209us) server_negotiation.cc:407] Connection header received
0811 03:16:33.439096 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:33.439101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:33.439185 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:33.439302 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:33.439858 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:33.441019 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:33.442608 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:33.443345 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:33.443484 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:33.444596 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:33.444605 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:33.444613 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:33.444641 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:33.444690 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:33.444713 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:33.444842 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:33.445081 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 03:16:33.445135 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 03:16:33.446151 1943 leader_election.cc:288] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:33.446286 1953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:33.448297 1943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:33.450623 1953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [term 1 LEADER]: Becoming Leader. State: Replica: 362c248afd3e467fb55079fb0b814ba2, State: Running, Role: LEADER
I20260811 03:16:33.451382 1953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c248afd3e467fb55079fb0b814ba2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260811 03:16:33.454697 1955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 362c248afd3e467fb55079fb0b814ba2. Latest consensus state: current_term: 1 leader_uuid: "362c248afd3e467fb55079fb0b814ba2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c248afd3e467fb55079fb0b814ba2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } } }
I20260811 03:16:33.455102 1955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:33.455618 1954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "362c248afd3e467fb55079fb0b814ba2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c248afd3e467fb55079fb0b814ba2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } } }
I20260811 03:16:33.455888 1954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2 [sys.catalog]: This master's current role is: LEADER
W20260811 03:16:33.463073 1972 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:16:33.463178 1972 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:16:33.463325 1973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:33.465494 1973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:33.474283 1973 catalog_manager.cc:1347] Generated new cluster ID: 80d1ead7f3dd45e18dc6368908bdd221
I20260811 03:16:33.474447 1973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:33.488873 1973 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:33.490351 1973 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:33.496944 1973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 362c248afd3e467fb55079fb0b814ba2: Generated new TSK 0
I20260811 03:16:33.497782 1973 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:33.512917 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 1868
W20260811 03:16:33.514228 1936 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59943
W20260811 03:16:33.515267 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 1868: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:16:33.515522 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 1868
W20260811 03:16:33.539906 1851 connection.cc:535] client connection to 127.0.0.1:38467 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38467): Connection reset by peer (error 104)
2026-08-11T03:16:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2107 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:33 dist-test-slave-tm1g krb5kdc[1979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:33 dist-test-slave-tm1g krb5kdc[1979](info): set up 2 sockets
Aug 11 03:16:33 dist-test-slave-tm1g krb5kdc[1979](info): commencing operation
krb5kdc: starting...
W20260811 03:16:34.716720 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:34 dist-test-slave-tm1g krb5kdc[1979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:35.029959 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32913
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:35.378216 1993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:35.378583 1993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:35.378763 1993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:35.386096 1993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:35.386210 1993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:35.386366 1993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:35.386466 1993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:35.386548 1993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:35.396552 1993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32913
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:35.398211 1993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:35.400095 1993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:35.412631 1998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:35.412559 1999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:35.412611 2001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:35.413615 1993 server_base.cc:1034] running on GCE node
I20260811 03:16:35.414505 1993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:35.415941 1993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:35.417174 1993 hybrid_clock.cc:648] HybridClock initialized: now 1786418195417126 us; error 41 us; skew 500 ppm
I20260811 03:16:35.417579 1993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:35 dist-test-slave-tm1g krb5kdc[1979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:35.425231 1993 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:35.426981 1993 webserver.cc:460] Webserver started at http://127.0.0.1:44333/ using document root <none> and password file <none>
I20260811 03:16:35.427723 1993 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:35.427870 1993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:35.428190 1993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:35.431340 1993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "95fbe08b740a4040858741f0d622a6e2"
format_stamp: "Formatted at 2026-08-11 03:16:35 on dist-test-slave-tm1g"
server_key: "105598e0b80e116c2358e7b61737daf1"
server_key_iv: "6add5d1f35617ba8ee43e3b0e51dfe1d"
server_key_version: "encryptionkey@0"
I20260811 03:16:35.432152 1993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "95fbe08b740a4040858741f0d622a6e2"
format_stamp: "Formatted at 2026-08-11 03:16:35 on dist-test-slave-tm1g"
server_key: "105598e0b80e116c2358e7b61737daf1"
server_key_iv: "6add5d1f35617ba8ee43e3b0e51dfe1d"
server_key_version: "encryptionkey@0"
I20260811 03:16:35.437559 1993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:16:35.441306 2009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:35.442976 1993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:16:35.443172 1993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "95fbe08b740a4040858741f0d622a6e2"
format_stamp: "Formatted at 2026-08-11 03:16:35 on dist-test-slave-tm1g"
server_key: "105598e0b80e116c2358e7b61737daf1"
server_key_iv: "6add5d1f35617ba8ee43e3b0e51dfe1d"
server_key_version: "encryptionkey@0"
I20260811 03:16:35.443398 1993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:35.449360 1993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:35.453722 1993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:35.454026 1993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:35.473480 1993 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32913
I20260811 03:16:35.473522 2065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32913 every 8 connection(s)
I20260811 03:16:35.475178 1993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:35.479046 2066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:35.480624 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 1993
I20260811 03:16:35.480825 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:35.481410 11504 external_mini_cluster.cc:1428] Setting key 3a7fb2ca92243b460972cd9c3d1df0db
I20260811 03:16:35.489601 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:35.483496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51821 (local address 127.0.0.1:32913)
0811 03:16:35.484387 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:35.484399 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:35.484447 (+ 48us) server_negotiation.cc:407] Connection header received
0811 03:16:35.486068 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:35.486092 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:35.486523 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:35.486872 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:35.487994 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:35.488001 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:35.488012 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:35.488036 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:35.488126 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:35.488149 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:35.488338 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:35.488442 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 03:16:35.488660 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":429,"threads_started":1}
I20260811 03:16:35.492995 2066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2: Bootstrap starting.
I20260811 03:16:35.493324 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:35.496235 2066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:35.497465 2066 log.cc:824] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:35.500536 2066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2: No bootstrap required, opened a new log
I20260811 03:16:35.500567 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:35.497703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54033 (local address 127.0.0.1:32913)
0811 03:16:35.497884 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:35.497890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:35.498660 (+ 770us) server_negotiation.cc:407] Connection header received
0811 03:16:35.498851 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:35.498861 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:35.498909 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:35.499026 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:35.499921 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:35.499927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:35.499931 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:35.499948 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:35.499999 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:35.500015 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:35.500130 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:35.500228 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 03:16:35.500286 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 03:16:35.510185 2066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fbe08b740a4040858741f0d622a6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32913 } }
I20260811 03:16:35.510687 2066 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:35.510797 2066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95fbe08b740a4040858741f0d622a6e2, State: Initialized, Role: FOLLOWER
I20260811 03:16:35.511437 2066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fbe08b740a4040858741f0d622a6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32913 } }
I20260811 03:16:35.511682 2066 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:35.511835 2066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:35.512018 2066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:35.514462 2066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fbe08b740a4040858741f0d622a6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32913 } }
I20260811 03:16:35.514990 2066 leader_election.cc:302] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95fbe08b740a4040858741f0d622a6e2; no voters:
I20260811 03:16:35.515666 2066 leader_election.cc:288] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:35.515928 2076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:35.517894 2066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:35.518477 2076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [term 1 LEADER]: Becoming Leader. State: Replica: 95fbe08b740a4040858741f0d622a6e2, State: Running, Role: LEADER
I20260811 03:16:35.519006 2076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fbe08b740a4040858741f0d622a6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32913 } }
I20260811 03:16:35.522480 2077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95fbe08b740a4040858741f0d622a6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fbe08b740a4040858741f0d622a6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32913 } } }
I20260811 03:16:35.522943 2077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:35.523509 2079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95fbe08b740a4040858741f0d622a6e2. Latest consensus state: current_term: 1 leader_uuid: "95fbe08b740a4040858741f0d622a6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fbe08b740a4040858741f0d622a6e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32913 } } }
I20260811 03:16:35.523850 2079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:35.527678 2089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:35.532362 2089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:35.541067 2089 catalog_manager.cc:1347] Generated new cluster ID: d8c08ea242984eb58ffb310e79c742d7
I20260811 03:16:35.541186 2089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:35.558154 2089 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:35.558692 2089 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:35.567040 2089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95fbe08b740a4040858741f0d622a6e2: Generated new TSK 0
I20260811 03:16:35.568128 2089 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:35.580765 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 1993
W20260811 03:16:35.582496 2060 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51821
W20260811 03:16:35.583209 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 1993: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:16:35.583350 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 1993
2026-08-11T03:16:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2063 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:35 dist-test-slave-tm1g krb5kdc[2099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:35 dist-test-slave-tm1g krb5kdc[2099](info): set up 2 sockets
Aug 11 03:16:35 dist-test-slave-tm1g krb5kdc[2099](info): commencing operation
krb5kdc: starting...
W20260811 03:16:36.810915 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.007s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:37 dist-test-slave-tm1g krb5kdc[2099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:37.142206 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33401
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:37.489017 2114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:37.489320 2114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:37.489415 2114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:37.496753 2114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:37.496861 2114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:37.496908 2114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:37.496963 2114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:37.497014 2114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:37.506922 2114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:37.508229 2114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:37.509521 2114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:37.521823 2119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:37.521792 2120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:37.521859 2122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:37.523116 2114 server_base.cc:1034] running on GCE node
I20260811 03:16:37.523949 2114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:37.525396 2114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:37.526618 2114 hybrid_clock.cc:648] HybridClock initialized: now 1786418197526562 us; error 41 us; skew 500 ppm
I20260811 03:16:37.527004 2114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:37 dist-test-slave-tm1g krb5kdc[2099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418197, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:37.530192 2114 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:37.531723 2114 webserver.cc:460] Webserver started at http://127.0.0.1:44793/ using document root <none> and password file <none>
I20260811 03:16:37.532418 2114 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:37.532565 2114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:37.532868 2114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:37.535945 2114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "6d81c8c412f84a569a06bdf417150e41"
format_stamp: "Formatted at 2026-08-11 03:16:37 on dist-test-slave-tm1g"
server_key: "f1ec09474462a27ee1ec0e310a166e00"
server_key_iv: "83f0cf4a2c98f43373926fdc68272cc8"
server_key_version: "encryptionkey@0"
I20260811 03:16:37.536739 2114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "6d81c8c412f84a569a06bdf417150e41"
format_stamp: "Formatted at 2026-08-11 03:16:37 on dist-test-slave-tm1g"
server_key: "f1ec09474462a27ee1ec0e310a166e00"
server_key_iv: "83f0cf4a2c98f43373926fdc68272cc8"
server_key_version: "encryptionkey@0"
I20260811 03:16:37.541953 2114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 03:16:37.545452 2131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:37.546850 2114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:16:37.547032 2114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "6d81c8c412f84a569a06bdf417150e41"
format_stamp: "Formatted at 2026-08-11 03:16:37 on dist-test-slave-tm1g"
server_key: "f1ec09474462a27ee1ec0e310a166e00"
server_key_iv: "83f0cf4a2c98f43373926fdc68272cc8"
server_key_version: "encryptionkey@0"
I20260811 03:16:37.547252 2114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:37.575042 2114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:37.579506 2114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:37.579793 2114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:37.599110 2114 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33401
I20260811 03:16:37.599153 2190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33401 every 8 connection(s)
I20260811 03:16:37.600557 2114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:37.604071 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 2114
I20260811 03:16:37.604283 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:37.604645 2191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:37.604871 11504 external_mini_cluster.cc:1428] Setting key dbc6236d6e488854cbc6241b203c442a
I20260811 03:16:37.619798 2191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41: Bootstrap starting.
Aug 11 03:16:37 dist-test-slave-tm1g krb5kdc[2099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 03:16:37.623831 2191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:37.625209 2191 log.cc:824] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:37.628952 2191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41: No bootstrap required, opened a new log
I20260811 03:16:37.630595 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:37.606787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59735 (local address 127.0.0.1:33401)
0811 03:16:37.607602 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:37.607613 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:37.607658 (+ 45us) server_negotiation.cc:407] Connection header received
0811 03:16:37.609562 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:37.609584 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:37.609997 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:37.610378 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:37.612237 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:37.613589 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:37.615546 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:37.616271 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:37.616413 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:37.621324 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:37.621336 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:37.621357 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:37.621413 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:37.625539 (+ 4126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:37.626589 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:37.626597 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:37.626603 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:37.626846 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:37.627281 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:37.627286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:37.627289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:37.628009 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:37.628321 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:37.628744 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 03:16:37.629192 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":357,"threads_started":1}
I20260811 03:16:37.634574 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:37.637790 2191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d81c8c412f84a569a06bdf417150e41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } }
I20260811 03:16:37.638156 2191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:37.638262 2191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d81c8c412f84a569a06bdf417150e41, State: Initialized, Role: FOLLOWER
I20260811 03:16:37.638917 2191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d81c8c412f84a569a06bdf417150e41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } }
I20260811 03:16:37.639216 2191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:37.639430 2191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:37.639627 2191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:37.642231 2191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d81c8c412f84a569a06bdf417150e41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } }
I20260811 03:16:37.642853 2191 leader_election.cc:302] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d81c8c412f84a569a06bdf417150e41; no voters:
I20260811 03:16:37.643563 2191 leader_election.cc:288] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:37.643672 2202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:37.644816 2202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [term 1 LEADER]: Becoming Leader. State: Replica: 6d81c8c412f84a569a06bdf417150e41, State: Running, Role: LEADER
I20260811 03:16:37.645485 2202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d81c8c412f84a569a06bdf417150e41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } }
I20260811 03:16:37.646057 2191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:37.648420 2205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d81c8c412f84a569a06bdf417150e41. Latest consensus state: current_term: 1 leader_uuid: "6d81c8c412f84a569a06bdf417150e41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d81c8c412f84a569a06bdf417150e41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } } }
I20260811 03:16:37.648880 2205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:37.649935 2204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d81c8c412f84a569a06bdf417150e41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d81c8c412f84a569a06bdf417150e41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33401 } } }
I20260811 03:16:37.650597 2204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:37.650768 2208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:37.658846 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:37.639195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45611 (local address 127.0.0.1:33401)
0811 03:16:37.639385 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:37.639393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:37.639807 (+ 414us) server_negotiation.cc:407] Connection header received
0811 03:16:37.640000 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:37.640008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:37.640091 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:37.640201 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:37.641798 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:37.642947 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:37.644596 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:37.645214 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:37.645290 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:37.648354 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:37.648363 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:37.648371 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:37.648423 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:37.653599 (+ 5176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:37.654800 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:37.654815 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:37.654821 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:37.655015 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:37.655631 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:37.655639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:37.655645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:37.656179 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:37.656647 (+ 468us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:37.658012 (+ 1365us) server_negotiation.cc:299] Negotiation successful
0811 03:16:37.658324 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:16:37.662568 2208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:37.672001 2208 catalog_manager.cc:1347] Generated new cluster ID: 8ffd66883c40482e8316fa993c80392b
I20260811 03:16:37.672153 2208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:37.684160 2208 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:37.685497 2208 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:37.693565 2208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d81c8c412f84a569a06bdf417150e41: Generated new TSK 0
I20260811 03:16:37.694651 2208 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:37.702013 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2114
I20260811 03:16:37.703169 2186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:37.797617 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 2114
2026-08-11T03:16:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2213 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:37 dist-test-slave-tm1g krb5kdc[2228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:37 dist-test-slave-tm1g krb5kdc[2228](info): set up 2 sockets
Aug 11 03:16:37 dist-test-slave-tm1g krb5kdc[2228](info): commencing operation
krb5kdc: starting...
W20260811 03:16:38.989531 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.010s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:39 dist-test-slave-tm1g krb5kdc[2228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:39.318194 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:39.664426 2242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:39.664768 2242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:39.664943 2242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:39.672220 2242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:39.672403 2242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:39.672508 2242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:39.672610 2242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:39.672684 2242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:39.682271 2242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:39.683630 2242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:39.684944 2242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:39.696831 2247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:39.696856 2250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:39.696808 2248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:39.697778 2242 server_base.cc:1034] running on GCE node
I20260811 03:16:39.698633 2242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:39.700047 2242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:39.701277 2242 hybrid_clock.cc:648] HybridClock initialized: now 1786418199701234 us; error 38 us; skew 500 ppm
I20260811 03:16:39.701634 2242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:39 dist-test-slave-tm1g krb5kdc[2228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:39.704885 2242 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:39.706389 2242 webserver.cc:460] Webserver started at http://127.0.0.1:38291/ using document root <none> and password file <none>
I20260811 03:16:39.707051 2242 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:39.707154 2242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:39.707402 2242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:39.710484 2242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "9b960b38bd1b4c3cbbbd12829debf103"
format_stamp: "Formatted at 2026-08-11 03:16:39 on dist-test-slave-tm1g"
server_key: "171ba3425bbb7115a23ed6b70ffc1ba6"
server_key_iv: "67985922ab09a88ef6028a93f0b00a1e"
server_key_version: "encryptionkey@0"
I20260811 03:16:39.711210 2242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "9b960b38bd1b4c3cbbbd12829debf103"
format_stamp: "Formatted at 2026-08-11 03:16:39 on dist-test-slave-tm1g"
server_key: "171ba3425bbb7115a23ed6b70ffc1ba6"
server_key_iv: "67985922ab09a88ef6028a93f0b00a1e"
server_key_version: "encryptionkey@0"
I20260811 03:16:39.716348 2242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:16:39.719826 2257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:39.721141 2242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:16:39.721284 2242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "9b960b38bd1b4c3cbbbd12829debf103"
format_stamp: "Formatted at 2026-08-11 03:16:39 on dist-test-slave-tm1g"
server_key: "171ba3425bbb7115a23ed6b70ffc1ba6"
server_key_iv: "67985922ab09a88ef6028a93f0b00a1e"
server_key_version: "encryptionkey@0"
I20260811 03:16:39.721448 2242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:39.737542 2242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:39.743005 2242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:39.743350 2242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:39.763494 2242 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44823
I20260811 03:16:39.763530 2311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44823 every 8 connection(s)
I20260811 03:16:39.764887 2242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:39.768946 2312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:39.768918 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 2242
I20260811 03:16:39.769117 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:39.769713 11504 external_mini_cluster.cc:1428] Setting key 3d31896871915b3f8814fc9d25d6318c
I20260811 03:16:39.783070 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:39.771609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45221 (local address 127.0.0.1:44823)
0811 03:16:39.772467 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:39.772479 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:39.772525 (+ 46us) server_negotiation.cc:407] Connection header received
0811 03:16:39.774067 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:39.774088 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:39.774534 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:39.774872 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:39.775469 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:39.776773 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:39.778615 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:39.779275 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:39.779406 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:39.780750 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:39.780758 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:39.780769 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:39.781330 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:39.781409 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:39.781435 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:39.781520 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:39.781763 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 03:16:39.782041 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":424,"threads_started":1}
I20260811 03:16:39.783159 2312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103: Bootstrap starting.
I20260811 03:16:39.786568 2312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:39.786791 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:39.787678 2312 log.cc:824] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:39.790654 2312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103: No bootstrap required, opened a new log
I20260811 03:16:39.797991 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:39.791323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41525 (local address 127.0.0.1:44823)
0811 03:16:39.791509 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:39.791516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:39.792021 (+ 505us) server_negotiation.cc:407] Connection header received
0811 03:16:39.792171 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:39.792176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:39.792261 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:39.792380 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:39.792839 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:39.793995 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:39.795340 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:39.796102 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:39.796237 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:39.797209 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:39.797219 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:39.797224 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:39.797245 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:39.797293 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:39.797320 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:39.797453 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:39.797562 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 03:16:39.797653 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 03:16:39.800091 2312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b960b38bd1b4c3cbbbd12829debf103" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } }
I20260811 03:16:39.800436 2312 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:39.800514 2312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b960b38bd1b4c3cbbbd12829debf103, State: Initialized, Role: FOLLOWER
I20260811 03:16:39.801002 2312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b960b38bd1b4c3cbbbd12829debf103" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } }
I20260811 03:16:39.801208 2312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:39.801306 2312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:39.801441 2312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:39.803649 2312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b960b38bd1b4c3cbbbd12829debf103" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } }
I20260811 03:16:39.804100 2312 leader_election.cc:302] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b960b38bd1b4c3cbbbd12829debf103; no voters:
I20260811 03:16:39.804673 2312 leader_election.cc:288] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:39.804823 2326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:39.806001 2326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [term 1 LEADER]: Becoming Leader. State: Replica: 9b960b38bd1b4c3cbbbd12829debf103, State: Running, Role: LEADER
I20260811 03:16:39.806608 2326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b960b38bd1b4c3cbbbd12829debf103" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } }
I20260811 03:16:39.807049 2312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:39.809486 2327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b960b38bd1b4c3cbbbd12829debf103" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b960b38bd1b4c3cbbbd12829debf103" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } } }
I20260811 03:16:39.809898 2327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:39.810268 2328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b960b38bd1b4c3cbbbd12829debf103. Latest consensus state: current_term: 1 leader_uuid: "9b960b38bd1b4c3cbbbd12829debf103" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b960b38bd1b4c3cbbbd12829debf103" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44823 } } }
I20260811 03:16:39.810601 2328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:39.813334 2336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:39.819074 2336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:39.828750 2336 catalog_manager.cc:1347] Generated new cluster ID: a3152a2f0cf84dab904106f44ce59b67
I20260811 03:16:39.828907 2336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:39.853168 2336 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:39.854688 2336 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:39.869035 2336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b960b38bd1b4c3cbbbd12829debf103: Generated new TSK 0
I20260811 03:16:39.870165 2336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:39.883284 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2242
W20260811 03:16:39.884413 2307 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45221
W20260811 03:16:39.885146 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2242: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:16:39.885286 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 2242
2026-08-11T03:16:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2087 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:40 dist-test-slave-tm1g krb5kdc[2349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:40 dist-test-slave-tm1g krb5kdc[2349](info): set up 2 sockets
Aug 11 03:16:40 dist-test-slave-tm1g krb5kdc[2349](info): commencing operation
krb5kdc: starting...
W20260811 03:16:41.123780 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:41 dist-test-slave-tm1g krb5kdc[2349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:41.454684 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:41.811302 2363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:41.811640 2363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:41.811844 2363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:41.819458 2363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:41.819583 2363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:41.819686 2363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:41.819794 2363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:41.819875 2363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:41.830435 2363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:41.831877 2363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:41.833428 2363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:41.846028 2368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:41.845957 2370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:41.846534 2372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:41.847033 2363 server_base.cc:1034] running on GCE node
I20260811 03:16:41.847811 2363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:41.849293 2363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:41.850510 2363 hybrid_clock.cc:648] HybridClock initialized: now 1786418201850454 us; error 63 us; skew 500 ppm
I20260811 03:16:41.850881 2363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:41 dist-test-slave-tm1g krb5kdc[2349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:41.859458 2363 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:41.861337 2363 webserver.cc:460] Webserver started at http://127.0.0.1:38871/ using document root <none> and password file <none>
I20260811 03:16:41.862094 2363 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:41.862241 2363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:41.862596 2363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:41.865782 2363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "c51c49806feb48dc9d1a9b014fe8b1ef"
format_stamp: "Formatted at 2026-08-11 03:16:41 on dist-test-slave-tm1g"
server_key: "e254678aa6652e7d895ed00f52be5fbc"
server_key_iv: "cbc265d899c4e58d344134cc9754587c"
server_key_version: "encryptionkey@0"
I20260811 03:16:41.866706 2363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "c51c49806feb48dc9d1a9b014fe8b1ef"
format_stamp: "Formatted at 2026-08-11 03:16:41 on dist-test-slave-tm1g"
server_key: "e254678aa6652e7d895ed00f52be5fbc"
server_key_iv: "cbc265d899c4e58d344134cc9754587c"
server_key_version: "encryptionkey@0"
I20260811 03:16:41.872156 2363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:16:41.875735 2379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:41.877110 2363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:16:41.877297 2363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "c51c49806feb48dc9d1a9b014fe8b1ef"
format_stamp: "Formatted at 2026-08-11 03:16:41 on dist-test-slave-tm1g"
server_key: "e254678aa6652e7d895ed00f52be5fbc"
server_key_iv: "cbc265d899c4e58d344134cc9754587c"
server_key_version: "encryptionkey@0"
I20260811 03:16:41.877521 2363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:41.883541 2363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:41.887908 2363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:41.888216 2363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:41.908679 2363 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42065
I20260811 03:16:41.908728 2433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42065 every 8 connection(s)
I20260811 03:16:41.910169 2363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:41.914054 2434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:41.915115 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 2363
I20260811 03:16:41.915341 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:41.915802 11504 external_mini_cluster.cc:1428] Setting key c87e4da08c4f0457a374fa2578947596
I20260811 03:16:41.925293 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:41.917852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45303 (local address 127.0.0.1:42065)
0811 03:16:41.918728 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:41.918743 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:41.918801 (+ 58us) server_negotiation.cc:407] Connection header received
0811 03:16:41.921102 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:41.921133 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:41.921637 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:41.922123 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:41.923141 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:41.923150 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:41.923166 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:41.923182 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:41.923247 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:41.923276 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:41.923634 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:41.923789 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 03:16:41.924023 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":379,"threads_started":1}
I20260811 03:16:41.929176 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:41.929227 2434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef: Bootstrap starting.
I20260811 03:16:41.932380 2434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:41.933442 2434 log.cc:824] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:41.936359 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:41.933798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45591 (local address 127.0.0.1:42065)
0811 03:16:41.934036 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:41.934045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:41.934439 (+ 394us) server_negotiation.cc:407] Connection header received
0811 03:16:41.934595 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:41.934601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:41.934673 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:41.934812 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:41.935687 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:41.935692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:41.935697 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:41.935715 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:41.935761 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:41.935775 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:41.935899 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:41.936017 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 03:16:41.936082 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 03:16:41.936864 2434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef: No bootstrap required, opened a new log
I20260811 03:16:41.945102 2434 raft_consensus.cc:348] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c49806feb48dc9d1a9b014fe8b1ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260811 03:16:41.945551 2434 raft_consensus.cc:374] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:41.945658 2434 raft_consensus.cc:729] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c51c49806feb48dc9d1a9b014fe8b1ef, State: Initialized, Role: FOLLOWER
I20260811 03:16:41.946221 2434 consensus_queue.cc:260] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c49806feb48dc9d1a9b014fe8b1ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260811 03:16:41.946527 2434 raft_consensus.cc:388] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:41.946676 2434 raft_consensus.cc:482] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:41.946856 2434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:41.949169 2434 raft_consensus.cc:504] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c49806feb48dc9d1a9b014fe8b1ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260811 03:16:41.949695 2434 leader_election.cc:302] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c51c49806feb48dc9d1a9b014fe8b1ef; no voters:
I20260811 03:16:41.950423 2434 leader_election.cc:288] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:41.950603 2445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:41.951470 2445 raft_consensus.cc:686] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [term 1 LEADER]: Becoming Leader. State: Replica: c51c49806feb48dc9d1a9b014fe8b1ef, State: Running, Role: LEADER
I20260811 03:16:41.952219 2445 consensus_queue.cc:237] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c49806feb48dc9d1a9b014fe8b1ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260811 03:16:41.952911 2434 sys_catalog.cc:564] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:41.955261 2447 sys_catalog.cc:455] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader c51c49806feb48dc9d1a9b014fe8b1ef. Latest consensus state: current_term: 1 leader_uuid: "c51c49806feb48dc9d1a9b014fe8b1ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c49806feb48dc9d1a9b014fe8b1ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } } }
I20260811 03:16:41.955416 2446 sys_catalog.cc:455] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c51c49806feb48dc9d1a9b014fe8b1ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c49806feb48dc9d1a9b014fe8b1ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } } }
I20260811 03:16:41.955701 2447 sys_catalog.cc:458] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:41.955701 2446 sys_catalog.cc:458] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:41.957837 2453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:41.962702 2453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:41.972059 2453 catalog_manager.cc:1347] Generated new cluster ID: b6dd47d22cbd47f7bb3fdad532d72ffb
I20260811 03:16:41.972186 2453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:41.986984 2453 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:41.987552 2453 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:41.998162 2453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c51c49806feb48dc9d1a9b014fe8b1ef: Generated new TSK 0
I20260811 03:16:41.999290 2453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:42.005882 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2363
W20260811 03:16:42.007233 2428 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45303
W20260811 03:16:42.007977 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2363: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:16:42.008128 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 2363
2026-08-11T03:16:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2123 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:42 dist-test-slave-tm1g krb5kdc[2469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:42 dist-test-slave-tm1g krb5kdc[2469](info): set up 2 sockets
Aug 11 03:16:42 dist-test-slave-tm1g krb5kdc[2469](info): commencing operation
krb5kdc: starting...
W20260811 03:16:43.388437 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:43 dist-test-slave-tm1g krb5kdc[2469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:43.732187 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44519
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:44.086796 2484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:44.087088 2484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:44.087260 2484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:44.094616 2484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:44.094727 2484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:44.094839 2484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:44.094947 2484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:44.095036 2484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:44.105245 2484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:44.106683 2484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:44.108027 2484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:44.120437 2490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:44.120503 2489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:44.120482 2492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:44.121423 2484 server_base.cc:1034] running on GCE node
I20260811 03:16:44.122287 2484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:44.123790 2484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:44.125046 2484 hybrid_clock.cc:648] HybridClock initialized: now 1786418204124997 us; error 58 us; skew 500 ppm
I20260811 03:16:44.125469 2484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:44 dist-test-slave-tm1g krb5kdc[2469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418204, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:44.128803 2484 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:44.130466 2484 webserver.cc:460] Webserver started at http://127.0.0.1:45905/ using document root <none> and password file <none>
I20260811 03:16:44.131137 2484 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:44.131248 2484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:44.131525 2484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:44.134644 2484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc"
format_stamp: "Formatted at 2026-08-11 03:16:44 on dist-test-slave-tm1g"
server_key: "e21248d0908f0368090762372f4123ac"
server_key_iv: "38c78a604e91dd7b5aa1b9d62d5e6c7c"
server_key_version: "encryptionkey@0"
I20260811 03:16:44.135377 2484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc"
format_stamp: "Formatted at 2026-08-11 03:16:44 on dist-test-slave-tm1g"
server_key: "e21248d0908f0368090762372f4123ac"
server_key_iv: "38c78a604e91dd7b5aa1b9d62d5e6c7c"
server_key_version: "encryptionkey@0"
I20260811 03:16:44.140547 2484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260811 03:16:44.144078 2500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:44.145430 2484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:16:44.145622 2484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc"
format_stamp: "Formatted at 2026-08-11 03:16:44 on dist-test-slave-tm1g"
server_key: "e21248d0908f0368090762372f4123ac"
server_key_iv: "38c78a604e91dd7b5aa1b9d62d5e6c7c"
server_key_version: "encryptionkey@0"
I20260811 03:16:44.145805 2484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:44.167357 2484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:44.171877 2484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:44.172199 2484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:44.191843 2484 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44519
I20260811 03:16:44.191862 2552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44519 every 8 connection(s)
I20260811 03:16:44.193322 2484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:44.196149 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 2484
I20260811 03:16:44.196365 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:44.196954 11504 external_mini_cluster.cc:1428] Setting key c83862fabaa52942232d481d056b0986
I20260811 03:16:44.197764 2553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 03:16:44 dist-test-slave-tm1g krb5kdc[2469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 03:16:44.213151 2553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc: Bootstrap starting.
I20260811 03:16:44.217571 2553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:44.219062 2553 log.cc:824] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:44.221974 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:44.198889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:38051 (local address 127.0.0.1:44519)
0811 03:16:44.199707 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:44.199720 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:44.199765 (+ 45us) server_negotiation.cc:407] Connection header received
0811 03:16:44.201326 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:44.201348 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:44.201756 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:44.202082 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:44.204042 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:44.205411 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:44.207216 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:44.207895 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:44.208027 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:44.213020 (+ 4993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:44.213031 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:44.213051 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:44.213096 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:44.217096 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:44.218007 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:44.218019 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:44.218029 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:44.218281 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:44.218774 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:44.218780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:44.218785 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:44.219494 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:44.219908 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:44.220320 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 03:16:44.220649 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":368,"threads_started":1}
I20260811 03:16:44.222657 2553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc: No bootstrap required, opened a new log
I20260811 03:16:44.225980 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:44.231117 2553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } }
I20260811 03:16:44.231650 2553 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:44.231791 2553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb1a2cee9e44c8ebfa8a7a64eb39fdc, State: Initialized, Role: FOLLOWER
I20260811 03:16:44.232463 2553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } }
I20260811 03:16:44.232717 2553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:44.232873 2553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:44.233052 2553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:44.235945 2553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } }
I20260811 03:16:44.236565 2553 leader_election.cc:302] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cb1a2cee9e44c8ebfa8a7a64eb39fdc; no voters:
I20260811 03:16:44.237267 2553 leader_election.cc:288] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:44.237399 2564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:44.239305 2553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:44.240764 2564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [term 1 LEADER]: Becoming Leader. State: Replica: 9cb1a2cee9e44c8ebfa8a7a64eb39fdc, State: Running, Role: LEADER
I20260811 03:16:44.241566 2564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } }
I20260811 03:16:44.245110 2566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cb1a2cee9e44c8ebfa8a7a64eb39fdc. Latest consensus state: current_term: 1 leader_uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } } }
I20260811 03:16:44.245568 2566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:44.246060 2565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb1a2cee9e44c8ebfa8a7a64eb39fdc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } } }
I20260811 03:16:44.246378 2565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:44.250785 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:44.230442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:49299 (local address 127.0.0.1:44519)
0811 03:16:44.230627 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:44.230632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:44.231195 (+ 563us) server_negotiation.cc:407] Connection header received
0811 03:16:44.231327 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:44.231331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:44.231412 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:44.231569 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:44.233227 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:44.234533 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:44.235825 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:44.236465 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:44.236642 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:44.240554 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:44.240563 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:44.240571 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:44.240626 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:44.245275 (+ 4649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:44.246487 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:44.246493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:44.246498 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:44.246685 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:44.247434 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:44.247443 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:44.247449 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:44.247968 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:44.249606 (+ 1638us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:44.249840 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 03:16:44.250047 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 03:16:44.252641 2576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:44.257942 2576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:44.267469 2576 catalog_manager.cc:1347] Generated new cluster ID: e01151fa08f14543b9769e2b93474cac
I20260811 03:16:44.267588 2576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:44.293082 2576 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:44.295158 2576 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:44.302796 2576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cb1a2cee9e44c8ebfa8a7a64eb39fdc: Generated new TSK 0
I20260811 03:16:44.303629 2576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:44.314702 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2484
I20260811 03:16:44.315896 2547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:44.411585 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 2484
W20260811 03:16:44.433480 2466 connection.cc:535] client connection to 127.0.0.1:44519 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44519): Connection reset by peer (error 104)
2026-08-11T03:16:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:44 dist-test-slave-tm1g krb5kdc[2590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:44 dist-test-slave-tm1g krb5kdc[2590](info): set up 2 sockets
Aug 11 03:16:44 dist-test-slave-tm1g krb5kdc[2590](info): commencing operation
krb5kdc: starting...
W20260811 03:16:45.621062 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:45 dist-test-slave-tm1g krb5kdc[2590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:45.947540 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:46.292539 2604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:46.292831 2604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:46.293043 2604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:46.300374 2604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:46.300539 2604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:46.300647 2604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:46.300753 2604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:46.300827 2604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:46.310791 2604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:46.312162 2604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:46.313535 2604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:46.325805 2614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:46.325850 2611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:46.325758 2612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:46.326651 2604 server_base.cc:1034] running on GCE node
I20260811 03:16:46.327451 2604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:46.328869 2604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:46.330093 2604 hybrid_clock.cc:648] HybridClock initialized: now 1786418206330041 us; error 31 us; skew 500 ppm
I20260811 03:16:46.330518 2604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:46 dist-test-slave-tm1g krb5kdc[2590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:46.333762 2604 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:46.335443 2604 webserver.cc:460] Webserver started at http://127.0.0.1:34831/ using document root <none> and password file <none>
I20260811 03:16:46.336160 2604 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:46.336319 2604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:46.336638 2604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:46.339875 2604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc"
format_stamp: "Formatted at 2026-08-11 03:16:46 on dist-test-slave-tm1g"
server_key: "13cdfed9c1aa5a64a17a416c8958ee7e"
server_key_iv: "a3deea47410dd1bf347ff5f01ce94998"
server_key_version: "encryptionkey@0"
I20260811 03:16:46.340675 2604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc"
format_stamp: "Formatted at 2026-08-11 03:16:46 on dist-test-slave-tm1g"
server_key: "13cdfed9c1aa5a64a17a416c8958ee7e"
server_key_iv: "a3deea47410dd1bf347ff5f01ce94998"
server_key_version: "encryptionkey@0"
I20260811 03:16:46.346101 2604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 03:16:46.351394 2621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:46.353320 2604 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 03:16:46.353612 2604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc"
format_stamp: "Formatted at 2026-08-11 03:16:46 on dist-test-slave-tm1g"
server_key: "13cdfed9c1aa5a64a17a416c8958ee7e"
server_key_iv: "a3deea47410dd1bf347ff5f01ce94998"
server_key_version: "encryptionkey@0"
I20260811 03:16:46.353955 2604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:46.370812 2604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:46.375303 2604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:46.375600 2604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:46.395112 2604 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35659
I20260811 03:16:46.395186 2673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35659 every 8 connection(s)
I20260811 03:16:46.396536 2604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:46.398440 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 2604
I20260811 03:16:46.398640 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:46.399104 11504 external_mini_cluster.cc:1428] Setting key 39e7d4f3eb80704e8b506b46a372c454
I20260811 03:16:46.401262 2674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:46.412155 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:46.401036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:33683 (local address 127.0.0.1:35659)
0811 03:16:46.401839 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:46.401850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:46.401889 (+ 39us) server_negotiation.cc:407] Connection header received
0811 03:16:46.403575 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:46.403595 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:46.403971 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:46.404383 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:46.404931 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:46.406266 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:46.407651 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:46.408314 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:46.408519 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:46.409862 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:46.409869 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:46.409881 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:46.410510 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:46.410593 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:46.410620 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:46.410803 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:46.411104 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 03:16:46.411257 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":361,"threads_started":1}
I20260811 03:16:46.413250 2674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc: Bootstrap starting.
I20260811 03:16:46.416009 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:46.416870 2674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:46.417964 2674 log.cc:824] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:46.421156 2674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc: No bootstrap required, opened a new log
I20260811 03:16:46.427774 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:46.420551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:54943 (local address 127.0.0.1:35659)
0811 03:16:46.420750 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:46.420756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:46.421149 (+ 393us) server_negotiation.cc:407] Connection header received
0811 03:16:46.421289 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:46.421295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:46.421376 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:46.421499 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:46.421997 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:46.423284 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:46.424640 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:46.425454 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:46.425607 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:46.426839 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:46.426848 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:46.426854 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:46.426883 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:46.426961 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:46.426985 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:46.427120 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:46.427339 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 03:16:46.427396 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 03:16:46.431468 2674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35659 } }
I20260811 03:16:46.431881 2674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:46.432008 2674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788d8d7ecf7b4fc7a2eebdf131a4c3dc, State: Initialized, Role: FOLLOWER
I20260811 03:16:46.432628 2674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35659 } }
I20260811 03:16:46.432914 2674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:46.433084 2674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:46.433311 2674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:46.435981 2674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35659 } }
I20260811 03:16:46.436669 2674 leader_election.cc:302] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 788d8d7ecf7b4fc7a2eebdf131a4c3dc; no voters:
I20260811 03:16:46.437438 2674 leader_election.cc:288] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:46.437533 2685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:46.438293 2685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [term 1 LEADER]: Becoming Leader. State: Replica: 788d8d7ecf7b4fc7a2eebdf131a4c3dc, State: Running, Role: LEADER
I20260811 03:16:46.438915 2685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35659 } }
I20260811 03:16:46.439795 2674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:46.441862 2687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 788d8d7ecf7b4fc7a2eebdf131a4c3dc. Latest consensus state: current_term: 1 leader_uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35659 } } }
I20260811 03:16:46.442370 2687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:46.445919 2686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788d8d7ecf7b4fc7a2eebdf131a4c3dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35659 } } }
I20260811 03:16:46.446417 2686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:46.448190 2693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:46.453325 2693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:46.463152 2693 catalog_manager.cc:1347] Generated new cluster ID: d0ba826b2bcf4ffeb767c9dd917b49fd
I20260811 03:16:46.463289 2693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:46.494745 2693 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:46.496157 2693 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:46.507308 2693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 788d8d7ecf7b4fc7a2eebdf131a4c3dc: Generated new TSK 0
I20260811 03:16:46.508247 2693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:46.514430 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2604
W20260811 03:16:46.515597 2669 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.78:33683
W20260811 03:16:46.516407 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2604: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:16:46.516569 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 2604
W20260811 03:16:46.537894 2588 connection.cc:535] client connection to 127.0.0.1:35659 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35659): Connection reset by peer (error 104)
2026-08-11T03:16:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2104 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:46 dist-test-slave-tm1g krb5kdc[2707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:46 dist-test-slave-tm1g krb5kdc[2707](info): set up 2 sockets
Aug 11 03:16:46 dist-test-slave-tm1g krb5kdc[2707](info): commencing operation
krb5kdc: starting...
W20260811 03:16:47.713927 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:47 dist-test-slave-tm1g krb5kdc[2707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:48.053922 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35689
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:48.436553 2722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:48.436923 2722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:48.437089 2722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:48.444348 2722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:48.444504 2722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:48.444613 2722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:48.444725 2722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:48.444813 2722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:48.456079 2722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:48.457448 2722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:48.458897 2722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:48.471223 2727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:48.471175 2728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:48.471774 2730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:48.472296 2722 server_base.cc:1034] running on GCE node
I20260811 03:16:48.473102 2722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:48.474685 2722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:48.475942 2722 hybrid_clock.cc:648] HybridClock initialized: now 1786418208475891 us; error 45 us; skew 500 ppm
I20260811 03:16:48.476305 2722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:48 dist-test-slave-tm1g krb5kdc[2707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418208, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:48.483542 2722 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:48.485293 2722 webserver.cc:460] Webserver started at http://127.0.0.1:39755/ using document root <none> and password file <none>
I20260811 03:16:48.486068 2722 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:48.486218 2722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:48.486646 2722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:48.489830 2722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "94c5985c48f74293a8c63b0b57035803"
format_stamp: "Formatted at 2026-08-11 03:16:48 on dist-test-slave-tm1g"
server_key: "b22085bb96d992fddd846813e0116ccb"
server_key_iv: "ee4ec2837fab4e42ff16778ac0029d2c"
server_key_version: "encryptionkey@0"
I20260811 03:16:48.490710 2722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "94c5985c48f74293a8c63b0b57035803"
format_stamp: "Formatted at 2026-08-11 03:16:48 on dist-test-slave-tm1g"
server_key: "b22085bb96d992fddd846813e0116ccb"
server_key_iv: "ee4ec2837fab4e42ff16778ac0029d2c"
server_key_version: "encryptionkey@0"
I20260811 03:16:48.496270 2722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:16:48.500005 2737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:48.501569 2722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:16:48.501766 2722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "94c5985c48f74293a8c63b0b57035803"
format_stamp: "Formatted at 2026-08-11 03:16:48 on dist-test-slave-tm1g"
server_key: "b22085bb96d992fddd846813e0116ccb"
server_key_iv: "ee4ec2837fab4e42ff16778ac0029d2c"
server_key_version: "encryptionkey@0"
I20260811 03:16:48.501988 2722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:48.508423 2722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:48.512650 2722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:48.512988 2722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:48.532668 2722 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35689
I20260811 03:16:48.532711 2793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35689 every 8 connection(s)
I20260811 03:16:48.534106 2722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:48.535632 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 2722
I20260811 03:16:48.535832 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:48.536278 11504 external_mini_cluster.cc:1428] Setting key 980aaf91bcf3b8d7f7ae4239ca3b46e1
I20260811 03:16:48.538530 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:48.544462 2797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:48.538324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:57283 (local address 127.0.0.1:35689)
0811 03:16:48.539155 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:48.539167 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:48.539207 (+ 40us) server_negotiation.cc:407] Connection header received
0811 03:16:48.540890 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:48.540916 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:48.541243 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:48.541635 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:48.542855 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:48.542861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:48.542873 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:48.542886 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:48.542955 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:48.542973 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:48.543232 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:48.543382 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 03:16:48.543566 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":387,"threads_started":1}
I20260811 03:16:48.548703 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:48.551246 2794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803: Bootstrap starting.
I20260811 03:16:48.554750 2794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:48.556099 2794 log.cc:824] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:48.556164 2797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:48.553499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:39311 (local address 127.0.0.1:35689)
0811 03:16:48.553710 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:48.553716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:48.554073 (+ 357us) server_negotiation.cc:407] Connection header received
0811 03:16:48.554227 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:48.554231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:48.554276 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:48.554463 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:48.555479 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:48.555483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:48.555487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:48.555504 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:48.555544 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:48.555558 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:48.555656 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:48.555812 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 03:16:48.555862 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:16:48.559134 2794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803: No bootstrap required, opened a new log
I20260811 03:16:48.567129 2794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c5985c48f74293a8c63b0b57035803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35689 } }
I20260811 03:16:48.567538 2794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:48.567678 2794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94c5985c48f74293a8c63b0b57035803, State: Initialized, Role: FOLLOWER
I20260811 03:16:48.568377 2794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c5985c48f74293a8c63b0b57035803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35689 } }
I20260811 03:16:48.568631 2794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:48.568797 2794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:48.568972 2794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:48.571326 2794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c5985c48f74293a8c63b0b57035803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35689 } }
I20260811 03:16:48.571872 2794 leader_election.cc:302] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94c5985c48f74293a8c63b0b57035803; no voters:
I20260811 03:16:48.572518 2794 leader_election.cc:288] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:48.572680 2804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:48.573782 2804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [term 1 LEADER]: Becoming Leader. State: Replica: 94c5985c48f74293a8c63b0b57035803, State: Running, Role: LEADER
I20260811 03:16:48.574741 2804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c5985c48f74293a8c63b0b57035803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35689 } }
I20260811 03:16:48.575105 2794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:48.578212 2806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94c5985c48f74293a8c63b0b57035803. Latest consensus state: current_term: 1 leader_uuid: "94c5985c48f74293a8c63b0b57035803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c5985c48f74293a8c63b0b57035803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35689 } } }
I20260811 03:16:48.578095 2805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94c5985c48f74293a8c63b0b57035803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c5985c48f74293a8c63b0b57035803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35689 } } }
I20260811 03:16:48.578641 2805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:48.578641 2806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:48.579691 2813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:48.585740 2813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:48.594224 2813 catalog_manager.cc:1347] Generated new cluster ID: 1153acd48e7f4d3d8d88c7b72a561ccd
I20260811 03:16:48.594372 2813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:48.611479 2813 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:48.611974 2813 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:48.624248 2813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94c5985c48f74293a8c63b0b57035803: Generated new TSK 0
I20260811 03:16:48.625365 2813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:48.635891 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2722
W20260811 03:16:48.637094 2789 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.78:57283
W20260811 03:16:48.637786 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2722: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:16:48.637943 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 2722
2026-08-11T03:16:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2122 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:48 dist-test-slave-tm1g krb5kdc[2827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:48 dist-test-slave-tm1g krb5kdc[2827](info): set up 2 sockets
Aug 11 03:16:48 dist-test-slave-tm1g krb5kdc[2827](info): commencing operation
krb5kdc: starting...
W20260811 03:16:49.832976 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:50 dist-test-slave-tm1g krb5kdc[2827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:50.171703 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40763
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:50.529286 2841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:50.529587 2841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:50.529686 2841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:50.536929 2841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:50.537040 2841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:50.537087 2841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:50.537151 2841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:50.537196 2841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:50.547241 2841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40763
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:50.548513 2841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:50.549799 2841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:50.562052 2846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:50.562023 2847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:50.562095 2850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:50.563148 2841 server_base.cc:1034] running on GCE node
I20260811 03:16:50.563946 2841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:50.565287 2841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:50.566543 2841 hybrid_clock.cc:648] HybridClock initialized: now 1786418210566506 us; error 61 us; skew 500 ppm
I20260811 03:16:50.566902 2841 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:50 dist-test-slave-tm1g krb5kdc[2827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418210, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:50.570456 2841 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:50.572096 2841 webserver.cc:460] Webserver started at http://127.0.0.1:43557/ using document root <none> and password file <none>
I20260811 03:16:50.572830 2841 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:50.572989 2841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:50.573331 2841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:50.576633 2841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "7f8d4aada0464c37b6d05df86dc07383"
format_stamp: "Formatted at 2026-08-11 03:16:50 on dist-test-slave-tm1g"
server_key: "4a356cc67da4bf1097a455a6793e3655"
server_key_iv: "0bb6c2ca08401bebfdb7453228d4559e"
server_key_version: "encryptionkey@0"
I20260811 03:16:50.577476 2841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "7f8d4aada0464c37b6d05df86dc07383"
format_stamp: "Formatted at 2026-08-11 03:16:50 on dist-test-slave-tm1g"
server_key: "4a356cc67da4bf1097a455a6793e3655"
server_key_iv: "0bb6c2ca08401bebfdb7453228d4559e"
server_key_version: "encryptionkey@0"
I20260811 03:16:50.583277 2841 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 03:16:50.587028 2858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:50.588398 2841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:16:50.588608 2841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "7f8d4aada0464c37b6d05df86dc07383"
format_stamp: "Formatted at 2026-08-11 03:16:50 on dist-test-slave-tm1g"
server_key: "4a356cc67da4bf1097a455a6793e3655"
server_key_iv: "0bb6c2ca08401bebfdb7453228d4559e"
server_key_version: "encryptionkey@0"
I20260811 03:16:50.588819 2841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:50.608522 2841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:50.612983 2841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:50.613288 2841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:50.632902 2841 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40763
I20260811 03:16:50.632951 2914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40763 every 8 connection(s)
I20260811 03:16:50.634377 2841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:50.635246 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 2841
I20260811 03:16:50.635509 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:50.636044 11504 external_mini_cluster.cc:1428] Setting key 601f46ec578e953abd8e7f8c53141c7f
I20260811 03:16:50.639367 2915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 03:16:50 dist-test-slave-tm1g krb5kdc[2827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 03:16:50.652856 2915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383: Bootstrap starting.
I20260811 03:16:50.656215 2915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:50.657284 2915 log.cc:824] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:50.660414 2915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383: No bootstrap required, opened a new log
I20260811 03:16:50.661003 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:50.638081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:36451 (local address 127.0.0.1:40763)
0811 03:16:50.638957 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:50.638974 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:50.639026 (+ 52us) server_negotiation.cc:407] Connection header received
0811 03:16:50.641407 (+ 2381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:50.641436 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:50.641958 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:50.642441 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:50.643872 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:50.646010 (+ 2138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:50.647388 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:50.648597 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:50.648817 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:50.653302 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:50.653311 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:50.653326 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:50.653365 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:50.656603 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:50.657456 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:50.657462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:50.657469 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:50.657639 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:50.658144 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:50.658148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:50.658150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:50.658784 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:50.659109 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:50.659682 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 03:16:50.659992 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":409,"threads_started":1}
I20260811 03:16:50.665239 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:50.669127 2915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8d4aada0464c37b6d05df86dc07383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260811 03:16:50.669581 2915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:50.669749 2915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8d4aada0464c37b6d05df86dc07383, State: Initialized, Role: FOLLOWER
I20260811 03:16:50.670646 2915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8d4aada0464c37b6d05df86dc07383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260811 03:16:50.670980 2915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:50.671150 2915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:50.671383 2915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:50.673697 2915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8d4aada0464c37b6d05df86dc07383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260811 03:16:50.674221 2915 leader_election.cc:302] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f8d4aada0464c37b6d05df86dc07383; no voters:
I20260811 03:16:50.675034 2915 leader_election.cc:288] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:50.675148 2926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:50.676772 2926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [term 1 LEADER]: Becoming Leader. State: Replica: 7f8d4aada0464c37b6d05df86dc07383, State: Running, Role: LEADER
I20260811 03:16:50.677392 2926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8d4aada0464c37b6d05df86dc07383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260811 03:16:50.677815 2915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:50.680156 2927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f8d4aada0464c37b6d05df86dc07383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8d4aada0464c37b6d05df86dc07383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } } }
I20260811 03:16:50.680563 2927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:50.681150 2928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f8d4aada0464c37b6d05df86dc07383. Latest consensus state: current_term: 1 leader_uuid: "7f8d4aada0464c37b6d05df86dc07383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8d4aada0464c37b6d05df86dc07383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } } }
I20260811 03:16:50.682785 2928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:50.686972 2933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:50.689646 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:50.669875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:45055 (local address 127.0.0.1:40763)
0811 03:16:50.670111 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:50.670120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:50.670608 (+ 488us) server_negotiation.cc:407] Connection header received
0811 03:16:50.670795 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:50.670798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:50.670861 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:50.670975 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:50.672513 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:50.673322 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:50.674788 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:50.675352 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:50.675522 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:50.679687 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:50.679697 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:50.679705 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:50.679766 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:50.685355 (+ 5589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:50.686580 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:50.686589 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:50.686594 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:50.686828 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:50.687496 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:50.687501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:50.687504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:50.688054 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:50.688484 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:50.688864 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 03:16:50.689128 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 03:16:50.701956 2933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:50.711071 2933 catalog_manager.cc:1347] Generated new cluster ID: bc455e0082f14cbe8340280e6d7f0ca2
I20260811 03:16:50.711247 2933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:50.744205 2933 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:50.745599 2933 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:50.758641 2933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f8d4aada0464c37b6d05df86dc07383: Generated new TSK 0
I20260811 03:16:50.759510 2933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:50.766983 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2841
I20260811 03:16:50.768203 2910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:50.864699 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 2841
W20260811 03:16:50.885775 2825 connection.cc:535] client connection to 127.0.0.1:40763 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40763): Connection reset by peer (error 104)
2026-08-11T03:16:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2230 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:50 dist-test-slave-tm1g krb5kdc[2950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:50 dist-test-slave-tm1g krb5kdc[2950](info): set up 2 sockets
Aug 11 03:16:50 dist-test-slave-tm1g krb5kdc[2950](info): commencing operation
krb5kdc: starting...
W20260811 03:16:52.086047 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:52 dist-test-slave-tm1g krb5kdc[2950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:52.442042 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38625
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:52.799703 2965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:52.800004 2965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:52.800225 2965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:52.807601 2965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:52.807720 2965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:52.807833 2965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:52.807917 2965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:52.807987 2965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:52.818336 2965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38625
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:52.819741 2965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:52.821159 2965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:52.833802 2971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:52.833815 2974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:52.833940 2972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:52.835249 2965 server_base.cc:1034] running on GCE node
I20260811 03:16:52.836094 2965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:52.837621 2965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:52.838852 2965 hybrid_clock.cc:648] HybridClock initialized: now 1786418212838807 us; error 40 us; skew 500 ppm
I20260811 03:16:52.839234 2965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:52 dist-test-slave-tm1g krb5kdc[2950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418212, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:52.842496 2965 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:52.844110 2965 webserver.cc:460] Webserver started at http://127.0.0.1:40901/ using document root <none> and password file <none>
I20260811 03:16:52.844820 2965 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:52.844964 2965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:52.845286 2965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:52.848393 2965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "38f0a690572d487885d5896b96904b52"
format_stamp: "Formatted at 2026-08-11 03:16:52 on dist-test-slave-tm1g"
server_key: "35e1dd5c9774df74443ae4a47360c49e"
server_key_iv: "28c48c3c212a464cf103b1b9aeb111df"
server_key_version: "encryptionkey@0"
I20260811 03:16:52.849200 2965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "38f0a690572d487885d5896b96904b52"
format_stamp: "Formatted at 2026-08-11 03:16:52 on dist-test-slave-tm1g"
server_key: "35e1dd5c9774df74443ae4a47360c49e"
server_key_iv: "28c48c3c212a464cf103b1b9aeb111df"
server_key_version: "encryptionkey@0"
I20260811 03:16:52.854739 2965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:16:52.858233 2981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:52.859649 2965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:16:52.859841 2965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "38f0a690572d487885d5896b96904b52"
format_stamp: "Formatted at 2026-08-11 03:16:52 on dist-test-slave-tm1g"
server_key: "35e1dd5c9774df74443ae4a47360c49e"
server_key_iv: "28c48c3c212a464cf103b1b9aeb111df"
server_key_version: "encryptionkey@0"
I20260811 03:16:52.860057 2965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:52.888299 2965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:52.892848 2965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:52.893186 2965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:52.913362 2965 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38625
I20260811 03:16:52.913455 3038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38625 every 8 connection(s)
I20260811 03:16:52.914938 2965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:52.916294 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 2965
I20260811 03:16:52.916507 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:52.917080 11504 external_mini_cluster.cc:1428] Setting key 1fcbf776bd5ef55e6e10ce8e594aeeb4
I20260811 03:16:52.919502 3039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:52.932968 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:52.919158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:55137 (local address 127.0.0.1:38625)
0811 03:16:52.919957 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:52.919972 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:52.920020 (+ 48us) server_negotiation.cc:407] Connection header received
0811 03:16:52.922467 (+ 2447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:52.922495 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:52.923040 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:52.923484 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:52.924008 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:52.926132 (+ 2124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:52.927510 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:52.928596 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:52.928774 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:52.930053 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:52.930064 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:52.930087 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:52.930890 (+ 803us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:52.930997 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:52.931032 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:52.931193 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:52.931464 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 03:16:52.931641 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":350,"threads_started":1}
I20260811 03:16:52.934216 3039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52: Bootstrap starting.
I20260811 03:16:52.936950 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:52.937604 3039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:52.938872 3039 log.cc:824] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:52.942498 3039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52: No bootstrap required, opened a new log
I20260811 03:16:52.949602 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:52.942512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:39975 (local address 127.0.0.1:38625)
0811 03:16:52.942732 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:52.942738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:52.943164 (+ 426us) server_negotiation.cc:407] Connection header received
0811 03:16:52.943315 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:52.943321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:52.943406 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:52.943519 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:52.944046 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:52.945291 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:52.946777 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:52.947442 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:52.947590 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:52.948734 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:52.948740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:52.948747 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:52.948778 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:52.948846 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:52.948866 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:52.949013 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:52.949199 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 03:16:52.949254 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 03:16:52.952387 3039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f0a690572d487885d5896b96904b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38625 } }
I20260811 03:16:52.952759 3039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:52.952833 3039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38f0a690572d487885d5896b96904b52, State: Initialized, Role: FOLLOWER
I20260811 03:16:52.953685 3039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f0a690572d487885d5896b96904b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38625 } }
I20260811 03:16:52.953902 3039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:52.954006 3039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:52.954139 3039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:52.956754 3039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f0a690572d487885d5896b96904b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38625 } }
I20260811 03:16:52.957338 3039 leader_election.cc:302] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38f0a690572d487885d5896b96904b52; no voters:
I20260811 03:16:52.958038 3039 leader_election.cc:288] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:52.958149 3052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:52.958921 3052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [term 1 LEADER]: Becoming Leader. State: Replica: 38f0a690572d487885d5896b96904b52, State: Running, Role: LEADER
I20260811 03:16:52.959506 3052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f0a690572d487885d5896b96904b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38625 } }
I20260811 03:16:52.960310 3039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:52.962330 3054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38f0a690572d487885d5896b96904b52. Latest consensus state: current_term: 1 leader_uuid: "38f0a690572d487885d5896b96904b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f0a690572d487885d5896b96904b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38625 } } }
I20260811 03:16:52.962734 3054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:52.962572 3053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38f0a690572d487885d5896b96904b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f0a690572d487885d5896b96904b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38625 } } }
I20260811 03:16:52.962916 3053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:52.967048 3059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:52.972026 3059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:52.981971 3059 catalog_manager.cc:1347] Generated new cluster ID: 39b273e50f8644fdb70572415232a365
I20260811 03:16:52.982105 3059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:53.002800 3059 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:53.004786 3059 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:53.014518 3059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38f0a690572d487885d5896b96904b52: Generated new TSK 0
I20260811 03:16:53.015425 3059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:53.029064 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2965
W20260811 03:16:53.030257 3033 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.78:55137
W20260811 03:16:53.031132 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 2965: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:16:53.031302 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 2965
W20260811 03:16:53.049808 2948 connection.cc:535] client connection to 127.0.0.1:38625 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38625): Connection reset by peer (error 104)
2026-08-11T03:16:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2159 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:53 dist-test-slave-tm1g krb5kdc[3073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:53 dist-test-slave-tm1g krb5kdc[3073](info): set up 2 sockets
Aug 11 03:16:53 dist-test-slave-tm1g krb5kdc[3073](info): commencing operation
krb5kdc: starting...
W20260811 03:16:54.257376 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:54 dist-test-slave-tm1g krb5kdc[3073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:16:54.602466 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35655
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:54.955778 3088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:54.956081 3088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:54.956274 3088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:54.963712 3088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:54.963850 3088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:54.963941 3088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:54.964056 3088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:54.964143 3088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:54.974514 3088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35655
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:54.975879 3088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:54.977290 3088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:54.989969 3093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:54.989930 3094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:54.990293 3096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:54.990795 3088 server_base.cc:1034] running on GCE node
I20260811 03:16:54.991658 3088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:54.993184 3088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:54.994410 3088 hybrid_clock.cc:648] HybridClock initialized: now 1786418214994344 us; error 52 us; skew 500 ppm
I20260811 03:16:54.994786 3088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:54 dist-test-slave-tm1g krb5kdc[3073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:54.998018 3088 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:16:54.999657 3088 webserver.cc:460] Webserver started at http://127.0.0.1:34299/ using document root <none> and password file <none>
I20260811 03:16:55.000401 3088 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:55.000567 3088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:55.000895 3088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:55.004053 3088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "0423cc80a8cc4371af814ee36c2b9926"
format_stamp: "Formatted at 2026-08-11 03:16:54 on dist-test-slave-tm1g"
server_key: "1291ba8eeb1a54a3fa17970ddaeae354"
server_key_iv: "bdb2e1b513be81184ed1eec862a5b052"
server_key_version: "encryptionkey@0"
I20260811 03:16:55.004863 3088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "0423cc80a8cc4371af814ee36c2b9926"
format_stamp: "Formatted at 2026-08-11 03:16:54 on dist-test-slave-tm1g"
server_key: "1291ba8eeb1a54a3fa17970ddaeae354"
server_key_iv: "bdb2e1b513be81184ed1eec862a5b052"
server_key_version: "encryptionkey@0"
I20260811 03:16:55.010803 3088 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 03:16:55.014532 3103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:55.015933 3088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:16:55.016131 3088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "0423cc80a8cc4371af814ee36c2b9926"
format_stamp: "Formatted at 2026-08-11 03:16:54 on dist-test-slave-tm1g"
server_key: "1291ba8eeb1a54a3fa17970ddaeae354"
server_key_iv: "bdb2e1b513be81184ed1eec862a5b052"
server_key_version: "encryptionkey@0"
I20260811 03:16:55.016369 3088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:55.022541 3088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:55.026772 3088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:55.027081 3088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:55.046531 3088 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35655
I20260811 03:16:55.046586 3157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35655 every 8 connection(s)
I20260811 03:16:55.048060 3088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:55.052008 3159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:55.055184 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 3088
I20260811 03:16:55.055382 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:55.055931 11504 external_mini_cluster.cc:1428] Setting key 38bb90a4c1307e89d03dbd27f0c0c97e
I20260811 03:16:55.064116 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:55.057897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:45633 (local address 127.0.0.1:35655)
0811 03:16:55.058799 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:55.058811 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:55.058855 (+ 44us) server_negotiation.cc:407] Connection header received
0811 03:16:55.060608 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:55.060634 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:55.061000 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:55.061338 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:55.062477 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:55.062484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:55.062495 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:55.062510 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:55.062582 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:55.062602 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:55.062782 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:55.062902 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 03:16:55.063151 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":395,"threads_started":1}
I20260811 03:16:55.065023 3159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926: Bootstrap starting.
I20260811 03:16:55.068114 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:55.068653 3159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:55.069679 3159 log.cc:824] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:55.072789 3159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926: No bootstrap required, opened a new log
I20260811 03:16:55.075557 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:55.072927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:50977 (local address 127.0.0.1:35655)
0811 03:16:55.073140 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:55.073147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:55.073496 (+ 349us) server_negotiation.cc:407] Connection header received
0811 03:16:55.073670 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:55.073674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:55.073719 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:55.073803 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:55.074841 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:55.074845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:55.074851 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:55.074866 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:55.074913 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:55.074927 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:55.075029 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:55.075164 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 03:16:55.075216 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 03:16:55.081211 3159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423cc80a8cc4371af814ee36c2b9926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } }
I20260811 03:16:55.081610 3159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:55.081720 3159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0423cc80a8cc4371af814ee36c2b9926, State: Initialized, Role: FOLLOWER
I20260811 03:16:55.082430 3159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423cc80a8cc4371af814ee36c2b9926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } }
I20260811 03:16:55.082705 3159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:55.082864 3159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:55.083076 3159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:55.085381 3159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423cc80a8cc4371af814ee36c2b9926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } }
I20260811 03:16:55.085894 3159 leader_election.cc:302] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0423cc80a8cc4371af814ee36c2b9926; no voters:
I20260811 03:16:55.086580 3159 leader_election.cc:288] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:55.086728 3170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:55.087517 3170 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [term 1 LEADER]: Becoming Leader. State: Replica: 0423cc80a8cc4371af814ee36c2b9926, State: Running, Role: LEADER
I20260811 03:16:55.088220 3170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423cc80a8cc4371af814ee36c2b9926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } }
I20260811 03:16:55.088917 3159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:55.091271 3172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0423cc80a8cc4371af814ee36c2b9926. Latest consensus state: current_term: 1 leader_uuid: "0423cc80a8cc4371af814ee36c2b9926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423cc80a8cc4371af814ee36c2b9926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } } }
I20260811 03:16:55.091718 3172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:55.091516 3171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0423cc80a8cc4371af814ee36c2b9926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423cc80a8cc4371af814ee36c2b9926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } } }
I20260811 03:16:55.091897 3171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:55.094184 3178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:55.099666 3178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:55.108676 3178 catalog_manager.cc:1347] Generated new cluster ID: f6a031603de64f3e9ae04f4ca7f87bbc
I20260811 03:16:55.108800 3178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:55.132025 3178 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:55.132685 3178 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:55.140246 3178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0423cc80a8cc4371af814ee36c2b9926: Generated new TSK 0
I20260811 03:16:55.141230 3178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:55.146016 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3088
W20260811 03:16:55.147210 3153 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.78:45633
W20260811 03:16:55.147995 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3088: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:16:55.148157 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 3088
2026-08-11T03:16:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2120 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:55 dist-test-slave-tm1g krb5kdc[3191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:55 dist-test-slave-tm1g krb5kdc[3191](info): set up 2 sockets
Aug 11 03:16:55 dist-test-slave-tm1g krb5kdc[3191](info): commencing operation
krb5kdc: starting...
W20260811 03:16:56.350785 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:56 dist-test-slave-tm1g krb5kdc[3191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:16:56.698405 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:45771
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:41951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:45771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:57.058413 3206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:57.058794 3206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:57.058961 3206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:57.066725 3206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:57.066898 3206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:57.067013 3206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:57.067116 3206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:57.067191 3206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:57.077708 3206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:41951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:45771
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:45771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:57.079253 3206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:57.080701 3206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:57.093607 3215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:57.093643 3212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:57.093703 3211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:57.094682 3206 server_base.cc:1034] running on GCE node
I20260811 03:16:57.095459 3206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:57.096926 3206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:57.098136 3206 hybrid_clock.cc:648] HybridClock initialized: now 1786418217098074 us; error 54 us; skew 500 ppm
I20260811 03:16:57.098555 3206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:57 dist-test-slave-tm1g krb5kdc[3191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418217, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:57.101851 3206 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:16:57.103354 3206 webserver.cc:460] Webserver started at http://127.11.60.62:37089/ using document root <none> and password file <none>
I20260811 03:16:57.104065 3206 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:57.104167 3206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:57.104424 3206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:57.107548 3206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "6c598b627dfd46b694a07f02b7a6252e"
format_stamp: "Formatted at 2026-08-11 03:16:57 on dist-test-slave-tm1g"
server_key: "a26d18f85fead3e79ee4b3e907346e0a"
server_key_iv: "57afd76f60319963c1311bd677b79586"
server_key_version: "encryptionkey@0"
I20260811 03:16:57.108256 3206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "6c598b627dfd46b694a07f02b7a6252e"
format_stamp: "Formatted at 2026-08-11 03:16:57 on dist-test-slave-tm1g"
server_key: "a26d18f85fead3e79ee4b3e907346e0a"
server_key_iv: "57afd76f60319963c1311bd677b79586"
server_key_version: "encryptionkey@0"
I20260811 03:16:57.113572 3206 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 03:16:57.117141 3222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:57.118561 3206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 03:16:57.118742 3206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "6c598b627dfd46b694a07f02b7a6252e"
format_stamp: "Formatted at 2026-08-11 03:16:57 on dist-test-slave-tm1g"
server_key: "a26d18f85fead3e79ee4b3e907346e0a"
server_key_iv: "57afd76f60319963c1311bd677b79586"
server_key_version: "encryptionkey@0"
I20260811 03:16:57.118942 3206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:57.141393 3206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:57.145555 3206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:57.145862 3206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:57.165251 3206 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:45771
I20260811 03:16:57.165274 3279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:45771 every 8 connection(s)
I20260811 03:16:57.166793 3206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:57.170722 3280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:57.172435 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 3206
I20260811 03:16:57.172665 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:57.173126 11504 external_mini_cluster.cc:1428] Setting key 884732d275c0f9cdb4ce99c32d1e4420
I20260811 03:16:57.186061 3280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e: Bootstrap starting.
Aug 11 03:16:57 dist-test-slave-tm1g krb5kdc[3191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:16:57.189558 3280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:57.191133 3280 log.cc:824] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:57.194828 3280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e: No bootstrap required, opened a new log
I20260811 03:16:57.199312 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:57.175121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38381 (local address 127.11.60.62:45771)
0811 03:16:57.176051 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:57.176066 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:57.176118 (+ 52us) server_negotiation.cc:407] Connection header received
0811 03:16:57.177897 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:57.177925 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:57.178541 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:57.179016 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:57.180377 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:57.182515 (+ 2138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:57.183923 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:57.185012 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:57.185214 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:57.189576 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:57.189596 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:57.189620 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:57.189669 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:57.193752 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:57.194816 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:57.194826 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:57.194834 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:57.195038 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:57.195624 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:57.195632 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:57.195637 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:57.196363 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:57.196713 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:57.197428 (+ 715us) server_negotiation.cc:299] Negotiation successful
0811 03:16:57.197815 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":408,"threads_started":1}
I20260811 03:16:57.203572 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:57.204937 3280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c598b627dfd46b694a07f02b7a6252e" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45771 } }
I20260811 03:16:57.205363 3280 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:57.205466 3280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c598b627dfd46b694a07f02b7a6252e, State: Initialized, Role: FOLLOWER
I20260811 03:16:57.206050 3280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c598b627dfd46b694a07f02b7a6252e" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45771 } }
I20260811 03:16:57.206311 3280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:57.206542 3280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:57.206743 3280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:57.209895 3280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c598b627dfd46b694a07f02b7a6252e" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45771 } }
I20260811 03:16:57.210484 3280 leader_election.cc:302] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c598b627dfd46b694a07f02b7a6252e; no voters:
I20260811 03:16:57.211190 3280 leader_election.cc:288] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:57.211310 3290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:57.212280 3290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [term 1 LEADER]: Becoming Leader. State: Replica: 6c598b627dfd46b694a07f02b7a6252e, State: Running, Role: LEADER
I20260811 03:16:57.212936 3290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c598b627dfd46b694a07f02b7a6252e" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45771 } }
I20260811 03:16:57.213469 3280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:57.215885 3292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c598b627dfd46b694a07f02b7a6252e. Latest consensus state: current_term: 1 leader_uuid: "6c598b627dfd46b694a07f02b7a6252e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c598b627dfd46b694a07f02b7a6252e" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45771 } } }
I20260811 03:16:57.216825 3292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:57.216514 3291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c598b627dfd46b694a07f02b7a6252e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c598b627dfd46b694a07f02b7a6252e" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45771 } } }
I20260811 03:16:57.217880 3291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:57.222342 3300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:57.227893 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:57.208351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55003 (local address 127.11.60.62:45771)
0811 03:16:57.208553 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:57.208557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:57.209014 (+ 457us) server_negotiation.cc:407] Connection header received
0811 03:16:57.209146 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:57.209151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:57.209224 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:57.209325 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:57.210841 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:57.212213 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:57.214049 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:57.214832 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:57.215004 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:57.219117 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:57.219127 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:57.219135 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:16:57.219190 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:57.224381 (+ 5191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:57.225443 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:57.225449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:57.225454 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:57.225607 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:16:57.226112 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:16:57.226116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:16:57.226118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:16:57.226517 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:57.226851 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:57.227290 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 03:16:57.227491 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 03:16:57.228593 3300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:57.238781 3300 catalog_manager.cc:1347] Generated new cluster ID: b612c0d766a74f70b4145d81d46ac1cf
I20260811 03:16:57.238909 3300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:57.269341 3300 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:57.270812 3300 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:57.277096 3300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c598b627dfd46b694a07f02b7a6252e: Generated new TSK 0
I20260811 03:16:57.278164 3300 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:57.284186 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3206
I20260811 03:16:57.285323 3273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:16:57.380525 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 3206
W20260811 03:16:57.401893 3189 connection.cc:535] client connection to 127.11.60.62:45771 recv error: Network error: failed to read from TLS socket (remote: 127.11.60.62:45771): Connection reset by peer (error 104)
2026-08-11T03:16:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2231 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:57 dist-test-slave-tm1g krb5kdc[3315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:57 dist-test-slave-tm1g krb5kdc[3315](info): set up 2 sockets
Aug 11 03:16:57 dist-test-slave-tm1g krb5kdc[3315](info): commencing operation
krb5kdc: starting...
W20260811 03:16:58.597550 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:16:58 dist-test-slave-tm1g krb5kdc[3315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:16:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:16:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:16:58.933391 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:45673
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:39081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:45673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:16:59.288151 3332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:16:59.288441 3332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:16:59.288645 3332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:16:59.296003 3332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:16:59.296161 3332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:16:59.296269 3332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:16:59.296371 3332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:16:59.296454 3332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:16:59.306756 3332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:39081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:45673
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:45673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:16:59.308177 3332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:16:59.309604 3332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:16:59.322598 3338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:59.322644 3339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:16:59.322610 3341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:16:59.323614 3332 server_base.cc:1034] running on GCE node
I20260811 03:16:59.324366 3332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:16:59.325899 3332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:16:59.327108 3332 hybrid_clock.cc:648] HybridClock initialized: now 1786418219327047 us; error 56 us; skew 500 ppm
I20260811 03:16:59.327423 3332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:16:59 dist-test-slave-tm1g krb5kdc[3315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418219, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:16:59.334581 3332 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:16:59.336206 3332 webserver.cc:460] Webserver started at http://127.11.60.62:40375/ using document root <none> and password file <none>
I20260811 03:16:59.336890 3332 fs_manager.cc:359] Metadata directory not provided
I20260811 03:16:59.336993 3332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:16:59.337260 3332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:16:59.340332 3332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "49d192355b7a43ecb3e21cb9cdf3af81"
format_stamp: "Formatted at 2026-08-11 03:16:59 on dist-test-slave-tm1g"
server_key: "93e8d7665c576e51729a60d0506f626e"
server_key_iv: "28cbfa003b57cc76cd68285dad85635a"
server_key_version: "encryptionkey@0"
I20260811 03:16:59.341059 3332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "49d192355b7a43ecb3e21cb9cdf3af81"
format_stamp: "Formatted at 2026-08-11 03:16:59 on dist-test-slave-tm1g"
server_key: "93e8d7665c576e51729a60d0506f626e"
server_key_iv: "28cbfa003b57cc76cd68285dad85635a"
server_key_version: "encryptionkey@0"
I20260811 03:16:59.346388 3332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:16:59.349857 3349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:16:59.351277 3332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 03:16:59.351465 3332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "49d192355b7a43ecb3e21cb9cdf3af81"
format_stamp: "Formatted at 2026-08-11 03:16:59 on dist-test-slave-tm1g"
server_key: "93e8d7665c576e51729a60d0506f626e"
server_key_iv: "28cbfa003b57cc76cd68285dad85635a"
server_key_version: "encryptionkey@0"
I20260811 03:16:59.351683 3332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:16:59.390468 3332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:16:59.395227 3332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:16:59.395560 3332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:16:59.415038 3332 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:45673
I20260811 03:16:59.415091 3407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:45673 every 8 connection(s)
I20260811 03:16:59.416508 3332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:16:59.416997 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 3332
I20260811 03:16:59.417234 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:16:59.417917 11504 external_mini_cluster.cc:1428] Setting key b9c2fd4c767d447b58b04afa7a454844
I20260811 03:16:59.421542 3408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:16:59.434039 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:59.419976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48091 (local address 127.11.60.62:45673)
0811 03:16:59.420810 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:59.420825 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:59.420876 (+ 51us) server_negotiation.cc:407] Connection header received
0811 03:16:59.423251 (+ 2375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:59.423279 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:59.423774 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:59.424206 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:59.424734 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:59.426822 (+ 2088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:59.428171 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:59.429252 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:59.429427 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:59.431064 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:59.431076 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:59.431095 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:59.431876 (+ 781us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:59.431991 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:59.432023 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:59.432190 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:59.432438 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 03:16:59.432651 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":335,"threads_started":1}
I20260811 03:16:59.436192 3408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81: Bootstrap starting.
I20260811 03:16:59.438046 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:16:59.439702 3408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81: Neither blocks nor log segments found. Creating new log.
I20260811 03:16:59.440776 3408 log.cc:824] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81: Log is configured to *not* fsync() on all Append() calls
I20260811 03:16:59.444382 3408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81: No bootstrap required, opened a new log
I20260811 03:16:59.449672 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:16:59.442596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50797 (local address 127.11.60.62:45673)
0811 03:16:59.442820 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 03:16:59.442829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:16:59.443257 (+ 428us) server_negotiation.cc:407] Connection header received
0811 03:16:59.443459 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:16:59.443464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:16:59.443541 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:16:59.443627 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:16:59.444260 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:59.445114 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:59.447083 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:16:59.447536 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:16:59.447646 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:16:59.448887 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:16:59.448892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:16:59.448895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:16:59.448915 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:16:59.448953 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:16:59.448970 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:16:59.449050 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 03:16:59.449255 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 03:16:59.449299 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 03:16:59.453645 3408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d192355b7a43ecb3e21cb9cdf3af81" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45673 } }
I20260811 03:16:59.454054 3408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:16:59.454175 3408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d192355b7a43ecb3e21cb9cdf3af81, State: Initialized, Role: FOLLOWER
I20260811 03:16:59.454833 3408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d192355b7a43ecb3e21cb9cdf3af81" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45673 } }
I20260811 03:16:59.455082 3408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:16:59.455238 3408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:16:59.455426 3408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:16:59.458118 3408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d192355b7a43ecb3e21cb9cdf3af81" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45673 } }
I20260811 03:16:59.458683 3408 leader_election.cc:302] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49d192355b7a43ecb3e21cb9cdf3af81; no voters:
I20260811 03:16:59.459266 3408 leader_election.cc:288] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:16:59.459414 3420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:16:59.460093 3420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [term 1 LEADER]: Becoming Leader. State: Replica: 49d192355b7a43ecb3e21cb9cdf3af81, State: Running, Role: LEADER
I20260811 03:16:59.460680 3420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d192355b7a43ecb3e21cb9cdf3af81" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45673 } }
I20260811 03:16:59.461531 3408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:16:59.463840 3421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49d192355b7a43ecb3e21cb9cdf3af81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d192355b7a43ecb3e21cb9cdf3af81" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45673 } } }
I20260811 03:16:59.464260 3421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:59.465940 3422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49d192355b7a43ecb3e21cb9cdf3af81. Latest consensus state: current_term: 1 leader_uuid: "49d192355b7a43ecb3e21cb9cdf3af81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d192355b7a43ecb3e21cb9cdf3af81" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45673 } } }
I20260811 03:16:59.466689 3430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:16:59.466805 3422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81 [sys.catalog]: This master's current role is: LEADER
I20260811 03:16:59.471207 3430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:16:59.481065 3430 catalog_manager.cc:1347] Generated new cluster ID: be1b4f378aa44bbcb48e8bcaeb8f2965
I20260811 03:16:59.481196 3430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:16:59.491763 3430 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:16:59.492995 3430 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:16:59.502116 3430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49d192355b7a43ecb3e21cb9cdf3af81: Generated new TSK 0
I20260811 03:16:59.503022 3430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:16:59.511706 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3332
W20260811 03:16:59.512890 3403 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48091
W20260811 03:16:59.513767 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3332: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:16:59.513919 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 3332
W20260811 03:16:59.535125 3313 connection.cc:535] client connection to 127.11.60.62:45673 recv error: Network error: failed to read from TLS socket (remote: 127.11.60.62:45673): Connection reset by peer (error 104)
2026-08-11T03:16:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2132 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:16:59 dist-test-slave-tm1g krb5kdc[3444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:16:59 dist-test-slave-tm1g krb5kdc[3444](info): set up 2 sockets
Aug 11 03:16:59 dist-test-slave-tm1g krb5kdc[3444](info): commencing operation
krb5kdc: starting...
W20260811 03:17:00.709514 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.011s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:17:00 dist-test-slave-tm1g krb5kdc[3444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:17:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:17:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:17:01.041476 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:45265
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:46101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:45265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:17:01.399861 3458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:17:01.400246 3458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:17:01.400353 3458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:17:01.408042 3458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:17:01.408164 3458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:17:01.408210 3458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:17:01.408278 3458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:17:01.408324 3458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:17:01.419275 3458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:46101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:45265
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:45265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:17:01.420675 3458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:17:01.422047 3458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:17:01.435125 3464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:01.435060 3465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:01.435091 3467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:17:01.435981 3458 server_base.cc:1034] running on GCE node
I20260811 03:17:01.436882 3458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:17:01.438482 3458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:17:01.439757 3458 hybrid_clock.cc:648] HybridClock initialized: now 1786418221439702 us; error 44 us; skew 500 ppm
I20260811 03:17:01.440156 3458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:17:01 dist-test-slave-tm1g krb5kdc[3444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418221, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:17:01.443439 3458 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:17:01.445067 3458 webserver.cc:460] Webserver started at http://127.11.60.62:33407/ using document root <none> and password file <none>
I20260811 03:17:01.445824 3458 fs_manager.cc:359] Metadata directory not provided
I20260811 03:17:01.445981 3458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:17:01.446364 3458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:17:01.449467 3458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "29c30de382074d269fe3032c352c0e41"
format_stamp: "Formatted at 2026-08-11 03:17:01 on dist-test-slave-tm1g"
server_key: "466af517b08cba1aac7fcd82e50f50d5"
server_key_iv: "93d05b95846473635fa82b1bf2cdfef4"
server_key_version: "encryptionkey@0"
I20260811 03:17:01.450286 3458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "29c30de382074d269fe3032c352c0e41"
format_stamp: "Formatted at 2026-08-11 03:17:01 on dist-test-slave-tm1g"
server_key: "466af517b08cba1aac7fcd82e50f50d5"
server_key_iv: "93d05b95846473635fa82b1bf2cdfef4"
server_key_version: "encryptionkey@0"
I20260811 03:17:01.455981 3458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 03:17:01.459789 3475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:17:01.461401 3458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:17:01.461603 3458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "29c30de382074d269fe3032c352c0e41"
format_stamp: "Formatted at 2026-08-11 03:17:01 on dist-test-slave-tm1g"
server_key: "466af517b08cba1aac7fcd82e50f50d5"
server_key_iv: "93d05b95846473635fa82b1bf2cdfef4"
server_key_version: "encryptionkey@0"
I20260811 03:17:01.461824 3458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:17:01.467815 3458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:17:01.471942 3458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:17:01.472268 3458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:17:01.492053 3458 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:45265
I20260811 03:17:01.492146 3531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:45265 every 8 connection(s)
I20260811 03:17:01.493625 3458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:17:01.494664 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 3458
I20260811 03:17:01.494906 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:17:01.495401 11504 external_mini_cluster.cc:1428] Setting key 6c40df3d9aa690308655e7a8cf257aff
I20260811 03:17:01.498225 3532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:17:01.503885 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:01.497275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33721 (local address 127.11.60.62:45265)
0811 03:17:01.498127 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:01.498139 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:01.498187 (+ 48us) server_negotiation.cc:407] Connection header received
0811 03:17:01.499951 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:01.499972 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:01.500289 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:01.500707 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:01.501803 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:01.501809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:01.501820 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:17:01.501834 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:01.501902 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:17:01.501923 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:01.502182 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:01.502337 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 03:17:01.502544 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":337,"threads_started":1}
I20260811 03:17:01.508312 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:17:01.512698 3532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41: Bootstrap starting.
I20260811 03:17:01.516738 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:01.513857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43651 (local address 127.11.60.62:45265)
0811 03:17:01.514087 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:01.514096 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:01.514657 (+ 561us) server_negotiation.cc:407] Connection header received
0811 03:17:01.514848 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:01.514852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:01.514905 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:01.514997 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:01.516018 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:01.516022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:01.516025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:17:01.516040 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:01.516085 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:17:01.516100 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:01.516192 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:01.516347 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 03:17:01.516400 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 03:17:01.517127 3532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41: Neither blocks nor log segments found. Creating new log.
I20260811 03:17:01.518564 3532 log.cc:824] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41: Log is configured to *not* fsync() on all Append() calls
I20260811 03:17:01.521412 3532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41: No bootstrap required, opened a new log
I20260811 03:17:01.529287 3532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c30de382074d269fe3032c352c0e41" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45265 } }
I20260811 03:17:01.529721 3532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:17:01.529856 3532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c30de382074d269fe3032c352c0e41, State: Initialized, Role: FOLLOWER
I20260811 03:17:01.530531 3532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c30de382074d269fe3032c352c0e41" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45265 } }
I20260811 03:17:01.530774 3532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:17:01.530957 3532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:17:01.531136 3532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:17:01.533391 3532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c30de382074d269fe3032c352c0e41" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45265 } }
I20260811 03:17:01.533910 3532 leader_election.cc:302] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29c30de382074d269fe3032c352c0e41; no voters:
I20260811 03:17:01.534735 3532 leader_election.cc:288] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:17:01.534848 3542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:17:01.535619 3542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [term 1 LEADER]: Becoming Leader. State: Replica: 29c30de382074d269fe3032c352c0e41, State: Running, Role: LEADER
I20260811 03:17:01.536206 3542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c30de382074d269fe3032c352c0e41" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45265 } }
I20260811 03:17:01.537050 3532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:17:01.539706 3543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29c30de382074d269fe3032c352c0e41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c30de382074d269fe3032c352c0e41" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45265 } } }
I20260811 03:17:01.540247 3543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:01.541428 3544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29c30de382074d269fe3032c352c0e41. Latest consensus state: current_term: 1 leader_uuid: "29c30de382074d269fe3032c352c0e41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c30de382074d269fe3032c352c0e41" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 45265 } } }
I20260811 03:17:01.541810 3544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:01.542505 3553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:17:01.547902 3553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:17:01.557125 3553 catalog_manager.cc:1347] Generated new cluster ID: 7574f28c18c74540a60fccb475e3eeb8
I20260811 03:17:01.557245 3553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:17:01.570267 3553 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:17:01.570724 3553 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:17:01.582515 3553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29c30de382074d269fe3032c352c0e41: Generated new TSK 0
I20260811 03:17:01.583402 3553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:17:01.594588 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3458
W20260811 03:17:01.595992 3527 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33721
W20260811 03:17:01.596761 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3458: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:17:01.596916 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 3458
2026-08-11T03:17:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2084 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:17:01 dist-test-slave-tm1g krb5kdc[3565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:17:01 dist-test-slave-tm1g krb5kdc[3565](info): set up 2 sockets
Aug 11 03:17:01 dist-test-slave-tm1g krb5kdc[3565](info): commencing operation
krb5kdc: starting...
W20260811 03:17:02.809578 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:17:03 dist-test-slave-tm1g krb5kdc[3565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:17:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:17:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:17:03.143590 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:35739
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:36453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:35739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:17:03.495580 3579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:17:03.495954 3579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:17:03.496134 3579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:17:03.504171 3579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:17:03.504333 3579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:17:03.504429 3579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:17:03.504545 3579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:17:03.504617 3579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:17:03.516357 3579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:36453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:35739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:35739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:17:03.517825 3579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:17:03.519322 3579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:17:03.532284 3584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:03.532236 3588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:03.532347 3585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:17:03.533187 3579 server_base.cc:1034] running on GCE node
I20260811 03:17:03.533986 3579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:17:03.535465 3579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:17:03.536677 3579 hybrid_clock.cc:648] HybridClock initialized: now 1786418223536636 us; error 33 us; skew 500 ppm
I20260811 03:17:03.536986 3579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:17:03 dist-test-slave-tm1g krb5kdc[3565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418223, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:17:03.545173 3579 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:17:03.547219 3579 webserver.cc:460] Webserver started at http://127.11.60.62:40745/ using document root <none> and password file <none>
I20260811 03:17:03.548146 3579 fs_manager.cc:359] Metadata directory not provided
I20260811 03:17:03.548280 3579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:17:03.548612 3579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:17:03.551769 3579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "c57b195a80394ed0a54ae5dcf1473354"
format_stamp: "Formatted at 2026-08-11 03:17:03 on dist-test-slave-tm1g"
server_key: "79dc1459377bbad8c7804e05a5b956ac"
server_key_iv: "c80e969713e9d9b4efca728cbbc69e41"
server_key_version: "encryptionkey@0"
I20260811 03:17:03.552572 3579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "c57b195a80394ed0a54ae5dcf1473354"
format_stamp: "Formatted at 2026-08-11 03:17:03 on dist-test-slave-tm1g"
server_key: "79dc1459377bbad8c7804e05a5b956ac"
server_key_iv: "c80e969713e9d9b4efca728cbbc69e41"
server_key_version: "encryptionkey@0"
I20260811 03:17:03.557914 3579 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:17:03.561421 3596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:17:03.562848 3579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:17:03.563040 3579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "c57b195a80394ed0a54ae5dcf1473354"
format_stamp: "Formatted at 2026-08-11 03:17:03 on dist-test-slave-tm1g"
server_key: "79dc1459377bbad8c7804e05a5b956ac"
server_key_iv: "c80e969713e9d9b4efca728cbbc69e41"
server_key_version: "encryptionkey@0"
I20260811 03:17:03.563241 3579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:17:03.584621 3579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:17:03.589344 3579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:17:03.589599 3579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:17:03.608806 3579 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:35739
I20260811 03:17:03.608841 3657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:35739 every 8 connection(s)
I20260811 03:17:03.610219 3579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:17:03.614054 3658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:17:03.616945 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 3579
I20260811 03:17:03.617143 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:17:03.617621 11504 external_mini_cluster.cc:1428] Setting key 53f63e731d5190f2edaa642f8f937c86
I20260811 03:17:03.627782 3658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354: Bootstrap starting.
I20260811 03:17:03.631628 3658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354: Neither blocks nor log segments found. Creating new log.
I20260811 03:17:03.632668 3658 log.cc:824] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354: Log is configured to *not* fsync() on all Append() calls
Aug 11 03:17:03 dist-test-slave-tm1g krb5kdc[3565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:17:03.636269 3658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354: No bootstrap required, opened a new log
I20260811 03:17:03.643671 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:03.619611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50665 (local address 127.11.60.62:35739)
0811 03:17:03.620450 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:03.620467 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:03.620521 (+ 54us) server_negotiation.cc:407] Connection header received
0811 03:17:03.622933 (+ 2412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:03.622960 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:03.623479 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:03.623994 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:03.625325 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:03.627330 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:03.628699 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:03.629846 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:03.630093 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:03.634538 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:03.634549 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:03.634568 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:03.634613 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:03.638780 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:03.639492 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:03.639500 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:03.639508 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:03.639735 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:03.640084 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:03.640095 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:03.640099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:03.640852 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:03.641330 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:03.641672 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 03:17:03.642212 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":365,"threads_started":1}
I20260811 03:17:03.646780 3658 raft_consensus.cc:348] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57b195a80394ed0a54ae5dcf1473354" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35739 } }
I20260811 03:17:03.647171 3658 raft_consensus.cc:374] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:17:03.647301 3658 raft_consensus.cc:729] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c57b195a80394ed0a54ae5dcf1473354, State: Initialized, Role: FOLLOWER
I20260811 03:17:03.647567 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:17:03.647943 3658 consensus_queue.cc:260] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57b195a80394ed0a54ae5dcf1473354" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35739 } }
I20260811 03:17:03.648190 3658 raft_consensus.cc:388] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:17:03.648319 3658 raft_consensus.cc:482] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:17:03.648506 3658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:17:03.651239 3658 raft_consensus.cc:504] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57b195a80394ed0a54ae5dcf1473354" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35739 } }
I20260811 03:17:03.651785 3658 leader_election.cc:302] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c57b195a80394ed0a54ae5dcf1473354; no voters:
I20260811 03:17:03.652503 3658 leader_election.cc:288] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:17:03.652688 3668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:17:03.653438 3668 raft_consensus.cc:686] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [term 1 LEADER]: Becoming Leader. State: Replica: c57b195a80394ed0a54ae5dcf1473354, State: Running, Role: LEADER
I20260811 03:17:03.654103 3668 consensus_queue.cc:237] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57b195a80394ed0a54ae5dcf1473354" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35739 } }
I20260811 03:17:03.655973 3668 sys_catalog.cc:455] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c57b195a80394ed0a54ae5dcf1473354. Latest consensus state: current_term: 1 leader_uuid: "c57b195a80394ed0a54ae5dcf1473354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57b195a80394ed0a54ae5dcf1473354" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35739 } } }
I20260811 03:17:03.656306 3668 sys_catalog.cc:458] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:03.656764 3671 sys_catalog.cc:455] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c57b195a80394ed0a54ae5dcf1473354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57b195a80394ed0a54ae5dcf1473354" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 35739 } } }
I20260811 03:17:03.657171 3671 sys_catalog.cc:458] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:03.661156 3658 sys_catalog.cc:564] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:17:03.661973 3674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:17:03.669552 3674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:17:03.673344 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:03.652452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59719 (local address 127.11.60.62:35739)
0811 03:17:03.652668 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:03.652674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:03.653353 (+ 679us) server_negotiation.cc:407] Connection header received
0811 03:17:03.653693 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:03.653701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:03.653808 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:03.653899 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:03.656749 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:03.658358 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:03.660450 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:03.661453 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:03.661610 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:03.665075 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:03.665082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:03.665087 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:03.665133 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:03.669224 (+ 4091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:03.670162 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:03.670168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:03.670172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:03.670338 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:03.670838 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:03.670848 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:03.670853 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:03.671303 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:03.671561 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:03.672701 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0811 03:17:03.672944 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 03:17:03.682530 3674 catalog_manager.cc:1347] Generated new cluster ID: 068908d098d04822883b268604a81792
I20260811 03:17:03.682646 3674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:17:03.704818 3674 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:17:03.707008 3674 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:17:03.717206 3674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c57b195a80394ed0a54ae5dcf1473354: Generated new TSK 0
I20260811 03:17:03.718375 3674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:17:03.727344 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3579
I20260811 03:17:03.729019 3653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:17:03.830031 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 3579
2026-08-11T03:17:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2231 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:17:03 dist-test-slave-tm1g krb5kdc[3692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:17:03 dist-test-slave-tm1g krb5kdc[3692](info): set up 2 sockets
Aug 11 03:17:03 dist-test-slave-tm1g krb5kdc[3692](info): commencing operation
krb5kdc: starting...
W20260811 03:17:05.017189 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.006s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:17:05 dist-test-slave-tm1g krb5kdc[3692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:17:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:17:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:17:05.358645 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:46045
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:37285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:46045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:17:05.720989 3709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:17:05.721359 3709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:17:05.721541 3709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:17:05.728874 3709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:17:05.729054 3709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:17:05.729168 3709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:17:05.729281 3709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:17:05.729373 3709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:17:05.739599 3709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:37285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:46045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:46045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:17:05.741571 3709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:17:05.743614 3709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:17:05.756894 3715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:05.756847 3716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:05.756870 3718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:17:05.757589 3709 server_base.cc:1034] running on GCE node
I20260811 03:17:05.758437 3709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:17:05.759904 3709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:17:05.761138 3709 hybrid_clock.cc:648] HybridClock initialized: now 1786418225761080 us; error 40 us; skew 500 ppm
I20260811 03:17:05.761587 3709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:17:05 dist-test-slave-tm1g krb5kdc[3692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:17:05.769060 3709 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:17:05.771031 3709 webserver.cc:460] Webserver started at http://127.11.60.62:37487/ using document root <none> and password file <none>
I20260811 03:17:05.771878 3709 fs_manager.cc:359] Metadata directory not provided
I20260811 03:17:05.772032 3709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:17:05.772395 3709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:17:05.775933 3709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "5d98284bd2024df19f9274cb81615df1"
format_stamp: "Formatted at 2026-08-11 03:17:05 on dist-test-slave-tm1g"
server_key: "c902befa8297120b3d9f59efbc5b9022"
server_key_iv: "0b0e99c9796a0ef7171955b26b4f0f4d"
server_key_version: "encryptionkey@0"
I20260811 03:17:05.776866 3709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "5d98284bd2024df19f9274cb81615df1"
format_stamp: "Formatted at 2026-08-11 03:17:05 on dist-test-slave-tm1g"
server_key: "c902befa8297120b3d9f59efbc5b9022"
server_key_iv: "0b0e99c9796a0ef7171955b26b4f0f4d"
server_key_version: "encryptionkey@0"
I20260811 03:17:05.783499 3709 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 03:17:05.787869 3725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:17:05.789507 3709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 03:17:05.789722 3709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "5d98284bd2024df19f9274cb81615df1"
format_stamp: "Formatted at 2026-08-11 03:17:05 on dist-test-slave-tm1g"
server_key: "c902befa8297120b3d9f59efbc5b9022"
server_key_iv: "0b0e99c9796a0ef7171955b26b4f0f4d"
server_key_version: "encryptionkey@0"
I20260811 03:17:05.789953 3709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:17:05.818724 3709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:17:05.823796 3709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:17:05.824152 3709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:17:05.843703 3709 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:46045
I20260811 03:17:05.843747 3786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:46045 every 8 connection(s)
I20260811 03:17:05.845191 3709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:17:05.849184 3787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:17:05.853374 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 3709
I20260811 03:17:05.853574 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:17:05.854020 11504 external_mini_cluster.cc:1428] Setting key e32894d0a8bd382117b573c59671ba08
I20260811 03:17:05.860636 3787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1: Bootstrap starting.
I20260811 03:17:05.864346 3787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1: Neither blocks nor log segments found. Creating new log.
I20260811 03:17:05.865777 3787 log.cc:824] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1: Log is configured to *not* fsync() on all Append() calls
I20260811 03:17:05.868793 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:05.855911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46563 (local address 127.11.60.62:46045)
0811 03:17:05.856742 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:05.856753 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:05.856798 (+ 45us) server_negotiation.cc:407] Connection header received
0811 03:17:05.858938 (+ 2140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:05.858961 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:05.859356 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:05.859822 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:05.860475 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:05.861920 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:05.863873 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:05.864649 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:05.864809 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:05.866125 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:05.866132 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:05.866146 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:17:05.867005 (+ 859us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:05.867114 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:17:05.867150 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:05.867280 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:05.867510 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 03:17:05.867816 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":389,"threads_started":1}
I20260811 03:17:05.869086 3787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1: No bootstrap required, opened a new log
I20260811 03:17:05.872623 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:17:05.878927 3787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d98284bd2024df19f9274cb81615df1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46045 } }
I20260811 03:17:05.879510 3787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:17:05.879616 3787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d98284bd2024df19f9274cb81615df1, State: Initialized, Role: FOLLOWER
I20260811 03:17:05.880328 3787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d98284bd2024df19f9274cb81615df1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46045 } }
I20260811 03:17:05.880686 3787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:17:05.880836 3787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:17:05.881018 3787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:17:05.883883 3787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d98284bd2024df19f9274cb81615df1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46045 } }
I20260811 03:17:05.884482 3787 leader_election.cc:302] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d98284bd2024df19f9274cb81615df1; no voters:
I20260811 03:17:05.885260 3787 leader_election.cc:288] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:17:05.885282 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:05.878127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58071 (local address 127.11.60.62:46045)
0811 03:17:05.878346 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:05.878352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:05.878917 (+ 565us) server_negotiation.cc:407] Connection header received
0811 03:17:05.879063 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:05.879068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:05.879151 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:05.879262 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:05.879945 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:05.881038 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:05.882488 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:05.883200 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:05.883325 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:05.884373 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:05.884378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:05.884383 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:17:05.884406 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:05.884451 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:17:05.884480 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:05.884579 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:05.884778 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 03:17:05.884872 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 03:17:05.885435 3797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:17:05.887187 3797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [term 1 LEADER]: Becoming Leader. State: Replica: 5d98284bd2024df19f9274cb81615df1, State: Running, Role: LEADER
I20260811 03:17:05.887849 3797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d98284bd2024df19f9274cb81615df1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46045 } }
I20260811 03:17:05.888327 3787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:17:05.891858 3798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d98284bd2024df19f9274cb81615df1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d98284bd2024df19f9274cb81615df1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46045 } } }
I20260811 03:17:05.891974 3799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d98284bd2024df19f9274cb81615df1. Latest consensus state: current_term: 1 leader_uuid: "5d98284bd2024df19f9274cb81615df1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d98284bd2024df19f9274cb81615df1" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 46045 } } }
I20260811 03:17:05.892402 3799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:05.892387 3798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:05.898679 3806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:17:05.903769 3806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:17:05.912947 3806 catalog_manager.cc:1347] Generated new cluster ID: 2d8e3286cc3649e88468bb92f2fdf587
I20260811 03:17:05.913065 3806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:17:05.938556 3806 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:17:05.940649 3806 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:17:05.953116 3806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d98284bd2024df19f9274cb81615df1: Generated new TSK 0
I20260811 03:17:05.953954 3806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:17:05.961225 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3709
W20260811 03:17:05.962646 3782 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46563
W20260811 03:17:05.963407 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3709: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:17:05.963567 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 3709
2026-08-11T03:17:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2133 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:17:06 dist-test-slave-tm1g krb5kdc[3819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:17:06 dist-test-slave-tm1g krb5kdc[3819](info): set up 2 sockets
Aug 11 03:17:06 dist-test-slave-tm1g krb5kdc[3819](info): commencing operation
krb5kdc: starting...
W20260811 03:17:07.160624 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.008s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:17:07 dist-test-slave-tm1g krb5kdc[3819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:17:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:17:07.491534 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42485
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:33647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:42485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:17:07.842595 3833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:17:07.842885 3833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:17:07.843050 3833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:17:07.850289 3833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:17:07.850512 3833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:17:07.850605 3833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:17:07.850697 3833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:17:07.850745 3833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:17:07.860435 3833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:33647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:42485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:42485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:17:07.861706 3833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:17:07.863122 3833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:17:07.875520 3838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:07.875423 3839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:07.875496 3841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:17:07.876281 3833 server_base.cc:1034] running on GCE node
I20260811 03:17:07.877077 3833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:17:07.878461 3833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:17:07.879678 3833 hybrid_clock.cc:648] HybridClock initialized: now 1786418227879624 us; error 52 us; skew 500 ppm
I20260811 03:17:07.879983 3833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:17:07 dist-test-slave-tm1g krb5kdc[3819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:17:07.883273 3833 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:17:07.884698 3833 webserver.cc:460] Webserver started at http://127.11.60.62:34017/ using document root <none> and password file <none>
I20260811 03:17:07.885362 3833 fs_manager.cc:359] Metadata directory not provided
I20260811 03:17:07.885479 3833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:17:07.885744 3833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:17:07.888757 3833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "e3952a9f5d534f10ab5175536e314854"
format_stamp: "Formatted at 2026-08-11 03:17:07 on dist-test-slave-tm1g"
server_key: "baa0638e4c4cd0284d641b28dec00aa1"
server_key_iv: "85a073d8e572ac7bfd5f28f9bcb9cb36"
server_key_version: "encryptionkey@0"
I20260811 03:17:07.889492 3833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "e3952a9f5d534f10ab5175536e314854"
format_stamp: "Formatted at 2026-08-11 03:17:07 on dist-test-slave-tm1g"
server_key: "baa0638e4c4cd0284d641b28dec00aa1"
server_key_iv: "85a073d8e572ac7bfd5f28f9bcb9cb36"
server_key_version: "encryptionkey@0"
I20260811 03:17:07.894784 3833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 03:17:07.898145 3849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:17:07.899483 3833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:17:07.899700 3833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "e3952a9f5d534f10ab5175536e314854"
format_stamp: "Formatted at 2026-08-11 03:17:07 on dist-test-slave-tm1g"
server_key: "baa0638e4c4cd0284d641b28dec00aa1"
server_key_iv: "85a073d8e572ac7bfd5f28f9bcb9cb36"
server_key_version: "encryptionkey@0"
I20260811 03:17:07.899907 3833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:17:07.905881 3833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:17:07.909829 3833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:17:07.910125 3833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:17:07.928946 3833 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:42485
I20260811 03:17:07.928992 3902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:42485 every 8 connection(s)
I20260811 03:17:07.930430 3833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:17:07.933810 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 3833
I20260811 03:17:07.934021 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:17:07.934505 3903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:17:07.934690 11504 external_mini_cluster.cc:1428] Setting key 908a49a46666fa02674e3102f4ea208b
I20260811 03:17:07.942863 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:07.936614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53867 (local address 127.11.60.62:42485)
0811 03:17:07.937401 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:07.937413 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:07.937456 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:17:07.939406 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:07.939428 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:07.939757 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:07.940176 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:07.941230 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:07.941236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:07.941248 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:17:07.941267 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:07.941322 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:17:07.941342 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:07.941601 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:07.941754 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 03:17:07.941920 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":358,"threads_started":1}
I20260811 03:17:07.946429 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:17:07.947816 3903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854: Bootstrap starting.
I20260811 03:17:07.951159 3903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854: Neither blocks nor log segments found. Creating new log.
I20260811 03:17:07.952553 3903 log.cc:824] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854: Log is configured to *not* fsync() on all Append() calls
I20260811 03:17:07.953860 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:07.950913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52517 (local address 127.11.60.62:42485)
0811 03:17:07.951117 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:07.951125 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:07.951761 (+ 636us) server_negotiation.cc:407] Connection header received
0811 03:17:07.951935 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:07.951938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:07.951983 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:07.952063 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:07.953135 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:07.953142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:07.953149 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:17:07.953172 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:07.953226 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:17:07.953254 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:07.953379 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:07.953489 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 03:17:07.953553 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 03:17:07.955760 3903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854: No bootstrap required, opened a new log
I20260811 03:17:07.963330 3903 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3952a9f5d534f10ab5175536e314854" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42485 } }
I20260811 03:17:07.963702 3903 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:17:07.963788 3903 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3952a9f5d534f10ab5175536e314854, State: Initialized, Role: FOLLOWER
I20260811 03:17:07.964339 3903 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3952a9f5d534f10ab5175536e314854" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42485 } }
I20260811 03:17:07.964547 3903 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:17:07.964650 3903 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:17:07.964833 3903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:17:07.967283 3903 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3952a9f5d534f10ab5175536e314854" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42485 } }
I20260811 03:17:07.967720 3903 leader_election.cc:302] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3952a9f5d534f10ab5175536e314854; no voters:
I20260811 03:17:07.968256 3903 leader_election.cc:288] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:17:07.968403 3914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:17:07.969133 3914 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [term 1 LEADER]: Becoming Leader. State: Replica: e3952a9f5d534f10ab5175536e314854, State: Running, Role: LEADER
I20260811 03:17:07.969720 3914 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3952a9f5d534f10ab5175536e314854" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42485 } }
I20260811 03:17:07.970503 3903 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:17:07.972505 3915 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3952a9f5d534f10ab5175536e314854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3952a9f5d534f10ab5175536e314854" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42485 } } }
I20260811 03:17:07.973022 3915 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:07.972530 3917 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3952a9f5d534f10ab5175536e314854. Latest consensus state: current_term: 1 leader_uuid: "e3952a9f5d534f10ab5175536e314854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3952a9f5d534f10ab5175536e314854" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 42485 } } }
I20260811 03:17:07.973615 3917 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:07.974400 3925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:17:07.979985 3925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:17:07.988840 3925 catalog_manager.cc:1347] Generated new cluster ID: 1fe19107391a496c9a650adb7aa81fde
I20260811 03:17:07.988968 3925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:17:08.001624 3925 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:17:08.002020 3925 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:17:08.013105 3925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3952a9f5d534f10ab5175536e314854: Generated new TSK 0
I20260811 03:17:08.013900 3925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:17:08.033036 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3833
W20260811 03:17:08.034972 3898 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53867
W20260811 03:17:08.035718 11504 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3833: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 03:17:08.035876 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 3833
2026-08-11T03:17:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2071 ms)
[----------] 18 tests from AuthTokenIssuingTest (38865 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:17:08 dist-test-slave-tm1g krb5kdc[3939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:17:08 dist-test-slave-tm1g krb5kdc[3939](info): set up 2 sockets
Aug 11 03:17:08 dist-test-slave-tm1g krb5kdc[3939](info): commencing operation
krb5kdc: starting...
W20260811 03:17:09.224833 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:17:09 dist-test-slave-tm1g krb5kdc[3939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:17:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:17:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:17:09.561372 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:39905
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:39189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:39905,127.11.60.61:32865,127.11.60.60:36527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:17:09.907637 3952 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:17:09.908028 3952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:17:09.908125 3952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:17:09.908270 3952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:17:09.915582 3952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:17:09.915664 3952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:17:09.915786 3952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:17:09.915874 3952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:17:09.915970 3952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:17:09.925784 3952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:39189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:39905,127.11.60.61:32865,127.11.60.60:36527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:39905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:17:09.927206 3952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:17:09.928542 3952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:17:09.941318 3957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:09.941277 3958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:09.941352 3960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:17:09.942509 3952 server_base.cc:1034] running on GCE node
I20260811 03:17:09.943449 3952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:17:09.944903 3952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:17:09.946178 3952 hybrid_clock.cc:648] HybridClock initialized: now 1786418229946112 us; error 38 us; skew 500 ppm
I20260811 03:17:09.946596 3952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:17:09 dist-test-slave-tm1g krb5kdc[3939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:17:09.949755 3952 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:17:09.951274 3952 webserver.cc:460] Webserver started at http://127.11.60.62:36897/ using document root <none> and password file <none>
I20260811 03:17:09.951996 3952 fs_manager.cc:359] Metadata directory not provided
I20260811 03:17:09.952138 3952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:17:09.952471 3952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:17:09.955634 3952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "cbb0ad4265e049d187fdec63477e9009"
format_stamp: "Formatted at 2026-08-11 03:17:09 on dist-test-slave-tm1g"
server_key: "64f4bf53de1237a03aaa13eff9198801"
server_key_iv: "200701d5f18f82f3edb087e76674b456"
server_key_version: "encryptionkey@0"
I20260811 03:17:09.956444 3952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "cbb0ad4265e049d187fdec63477e9009"
format_stamp: "Formatted at 2026-08-11 03:17:09 on dist-test-slave-tm1g"
server_key: "64f4bf53de1237a03aaa13eff9198801"
server_key_iv: "200701d5f18f82f3edb087e76674b456"
server_key_version: "encryptionkey@0"
I20260811 03:17:09.961592 3952 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 03:17:09.965056 3967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:17:09.966382 3952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:17:09.966583 3952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "cbb0ad4265e049d187fdec63477e9009"
format_stamp: "Formatted at 2026-08-11 03:17:09 on dist-test-slave-tm1g"
server_key: "64f4bf53de1237a03aaa13eff9198801"
server_key_iv: "200701d5f18f82f3edb087e76674b456"
server_key_version: "encryptionkey@0"
I20260811 03:17:09.966806 3952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:17:09.981668 3952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:17:09.985523 3952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:17:09.985817 3952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:17:10.004937 3952 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:39905
I20260811 03:17:10.004961 4021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:39905 every 8 connection(s)
I20260811 03:17:10.006343 3952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:17:10.010082 4022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:17:10.013530 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 3952
I20260811 03:17:10.013760 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:17:10.014474 11504 external_mini_cluster.cc:1428] Setting key 4ede9579f4381d8a108039c5d333a22b
I20260811 03:17:10.016784 4022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:17:10 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:17:10.042207 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.019952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39905 (local address 127.0.0.1:59354)
0811 03:17:10.020910 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.020975 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:10.023802 (+ 2827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:10.024661 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:10.024671 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:10.025493 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:10.026008 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:10.026035 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.027704 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.027712 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:10.029026 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:10.029039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.030446 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.030452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:10.030583 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:10.031261 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:10.031282 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:10.033651 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:10.038135 (+ 4484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:10.038146 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:10.038155 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:10.038626 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:10.039107 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:10.039110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:10.039112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:10.039275 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:10.040297 (+ 1022us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:10.040303 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:10.040504 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 03:17:10.040740 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 03:17:10.041113 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":447,"threads_started":1}
I20260811 03:17:10.042240 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.019952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.11.60.62:39905)
0811 03:17:10.021853 (+ 1901us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:10.021860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:10.021904 (+ 44us) server_negotiation.cc:407] Connection header received
0811 03:17:10.024185 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:10.024191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:10.024411 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:10.024578 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:10.026236 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.027541 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.029184 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.030162 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.030379 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:10.033894 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:10.033906 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:10.033924 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:10.033975 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:10.038000 (+ 4025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:10.038760 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:10.038771 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:10.038778 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:10.038995 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:10.039397 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:10.039405 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:10.039410 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:10.040164 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:10.040375 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:10.040922 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 03:17:10.041332 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":1716,"spinlock_wait_cycles":25472,"thread_start_us":1501,"threads_started":1}
I20260811 03:17:10.045188 4022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:17:10.046423 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.045882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58198)
0811 03:17:10.046092 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.046232 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 03:17:10.046717 3972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
I20260811 03:17:10.048378 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.047930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58200)
0811 03:17:10.048114 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.048204 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 03:17:10.048610 4022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:32865: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.60.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab.
Entry for principal kudu/127.11.60.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab.
I20260811 03:17:10.090700 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.61:32865
--webserver_interface=127.11.60.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab
--principal=kudu/127.11.60.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:39189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:39905,127.11.60.61:32865,127.11.60.60:36527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:17:10.097975 4022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } attempt: 1
I20260811 03:17:10.099263 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.098806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58204)
0811 03:17:10.099019 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.099114 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 03:17:10.100274 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.099917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58214)
0811 03:17:10.100072 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.100149 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 03:17:10.100441 4022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:32865: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
I20260811 03:17:10.168717 4022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } attempt: 2
I20260811 03:17:10.170127 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.169464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58218)
0811 03:17:10.169756 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.169891 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 03:17:10.171298 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.170852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58224)
0811 03:17:10.171020 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.171114 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 03:17:10.171583 4022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:32865: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
I20260811 03:17:10.262884 4022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } attempt: 3
I20260811 03:17:10.264302 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.263632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58240)
0811 03:17:10.263924 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.264077 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 03:17:10.265663 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.265136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58250)
0811 03:17:10.265339 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.265449 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 03:17:10.265908 4022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:32865: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
I20260811 03:17:10.409212 4022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } attempt: 4
I20260811 03:17:10.410689 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.409980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58252)
0811 03:17:10.410362 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.410505 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260811 03:17:10.411847 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.411390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58254)
0811 03:17:10.411572 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.411672 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 03:17:10.412119 4022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:32865: Network error: Client connection negotiation failed: client connection to 127.11.60.61:32865: connect: Connection refused (error 111)
W20260811 03:17:10.442996 4030 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:17:10.443360 4030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:17:10.443490 4030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:17:10.443626 4030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:17:10.451239 4030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:17:10.451341 4030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:17:10.451500 4030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:17:10.451660 4030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:17:10.451785 4030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:17:10.462169 4030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:39189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:39905,127.11.60.61:32865,127.11.60.60:36527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab
--principal=kudu/127.11.60.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.61:32865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.60.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:17:10.463593 4030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:17:10.465108 4030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:17:10.477557 4036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:10.477763 4035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:10.477722 4038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:17:10.478582 4030 server_base.cc:1034] running on GCE node
I20260811 03:17:10.479461 4030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:17:10.481055 4030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:17:10.482472 4030 hybrid_clock.cc:648] HybridClock initialized: now 1786418230482372 us; error 89 us; skew 500 ppm
I20260811 03:17:10.482848 4030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:17:10 dist-test-slave-tm1g krb5kdc[3939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418230, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:17:10.486392 4030 init.cc:377] Logged in from keytab as kudu/127.11.60.61@KRBTEST.COM (short username kudu)
I20260811 03:17:10.488560 4030 webserver.cc:460] Webserver started at http://127.11.60.61:44125/ using document root <none> and password file <none>
I20260811 03:17:10.489331 4030 fs_manager.cc:359] Metadata directory not provided
I20260811 03:17:10.489426 4030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:17:10.489794 4030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:17:10.493040 4030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/data/instance:
uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6"
format_stamp: "Formatted at 2026-08-11 03:17:10 on dist-test-slave-tm1g"
server_key: "22b974274df4b2b188decf99da4b4f1f"
server_key_iv: "dd25b47c95eb23eb5fc6ec4320af394c"
server_key_version: "encryptionkey@0"
I20260811 03:17:10.493899 4030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/wal/instance:
uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6"
format_stamp: "Formatted at 2026-08-11 03:17:10 on dist-test-slave-tm1g"
server_key: "22b974274df4b2b188decf99da4b4f1f"
server_key_iv: "dd25b47c95eb23eb5fc6ec4320af394c"
server_key_version: "encryptionkey@0"
I20260811 03:17:10.499529 4030 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 03:17:10.503171 4045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:17:10.504586 4030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:17:10.504786 4030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/wal
uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6"
format_stamp: "Formatted at 2026-08-11 03:17:10 on dist-test-slave-tm1g"
server_key: "22b974274df4b2b188decf99da4b4f1f"
server_key_iv: "dd25b47c95eb23eb5fc6ec4320af394c"
server_key_version: "encryptionkey@0"
I20260811 03:17:10.505010 4030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:17:10.524094 4030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:17:10.528307 4030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:17:10.528621 4030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:17:10.548987 4030 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.61:32865
I20260811 03:17:10.548998 4100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.61:32865 every 8 connection(s)
I20260811 03:17:10.550479 4030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
I20260811 03:17:10.554327 4101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:17:10.555097 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 4030
I20260811 03:17:10.555301 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-1/wal/instance
I20260811 03:17:10.555774 11504 external_mini_cluster.cc:1428] Setting key 08935e0d67de989ba2f4e5b3f0616535
I20260811 03:17:10.561137 4101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:17:10 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418230, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:17:10.581967 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.563833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59364 (local address 127.11.60.62:39905)
0811 03:17:10.564360 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:10.564364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:10.565334 (+ 970us) server_negotiation.cc:407] Connection header received
0811 03:17:10.567331 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:10.567337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:10.567437 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:10.567568 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:10.569106 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.569898 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.571678 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.572110 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.572252 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:10.576057 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:10.576062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:10.576065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:10.576108 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:10.578861 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:10.579613 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:10.579619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:10.579621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:10.579739 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:10.580187 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:10.580192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:10.580197 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:10.580485 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:10.580648 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:10.581354 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 03:17:10.581592 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":301,"threads_started":1}
I20260811 03:17:10.583047 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.564391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39905 (local address 127.0.0.1:59364)
0811 03:17:10.565154 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.565198 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:10.567056 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:10.567806 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:10.567815 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:10.568336 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:10.568885 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:10.568901 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.570075 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.570082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:10.571539 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:10.571552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.572339 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.572346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:10.572590 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:10.573297 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:10.573319 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:10.575805 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:10.579009 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:10.579018 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:10.579043 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:10.579468 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:10.579853 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:10.579857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:10.579861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:10.580064 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:10.580620 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:10.580629 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:10.580986 (+ 357us) client_negotiation.cc:769] Sending connection context
0811 03:17:10.581264 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 03:17:10.581786 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":335,"threads_started":1}
I20260811 03:17:10.584829 4101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.60.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.60@KRBTEST.COM" created.
Aug 11 03:17:10 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418230, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
I20260811 03:17:10.599823 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.585539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58266)
0811 03:17:10.585775 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.585790 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:10.585873 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:10.587180 (+ 1307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:10.587184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:10.587646 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:10.587776 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:10.587784 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.589241 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.589248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:10.590236 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:10.590248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.591145 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.591150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:10.591385 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:10.591985 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:10.592001 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:10.594022 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:10.597277 (+ 3255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:10.597284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:10.597288 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:10.597641 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:10.598014 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:10.598020 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:10.598024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:10.598166 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:10.598797 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:10.598805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:10.598967 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 03:17:10.599110 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 03:17:10.599396 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":45}
I20260811 03:17:10.599884 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.585830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58266 (local address 127.11.60.61:32865)
0811 03:17:10.586708 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:10.586714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:10.586741 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:17:10.586784 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:10.586789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:10.587028 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:10.587190 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:10.587977 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.589069 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.590447 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.590991 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.591137 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:10.594230 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:10.594241 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:10.594254 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:10.594286 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:10.597131 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:10.597781 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:10.597794 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:10.597796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:10.597906 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:10.598279 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:10.598287 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:10.598290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:10.598672 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:10.598899 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:10.599282 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 03:17:10.599510 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":729,"thread_start_us":557,"threads_started":1}
I20260811 03:17:10.602432 4101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:17:10.603754 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.603298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55844)
0811 03:17:10.603490 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.603602 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 03:17:10.604038 4048 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
I20260811 03:17:10.605619 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.605284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55854)
0811 03:17:10.605428 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.605520 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 03:17:10.605947 4101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:36527: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab.
Entry for principal kudu/127.11.60.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab.
I20260811 03:17:10.630069 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.60:36527
--webserver_interface=127.11.60.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab
--principal=kudu/127.11.60.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:39189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:39905,127.11.60.61:32865,127.11.60.60:36527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:17:10.655272 4101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } attempt: 1
I20260811 03:17:10.656517 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.655978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55866)
0811 03:17:10.656181 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.656295 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 03:17:10.657565 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.657208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55878)
0811 03:17:10.657383 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.657469 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 03:17:10.657848 4101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:36527: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
I20260811 03:17:10.711403 4022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } attempt: 5
Aug 11 03:17:10 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
I20260811 03:17:10.725984 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.712247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58278)
0811 03:17:10.712497 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.712521 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:10.712668 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:10.713073 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:10.713077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:10.713542 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:10.713721 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:10.713730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.714873 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.714883 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:10.715850 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:10.715863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.716619 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.716625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:10.716851 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:10.717569 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:10.717587 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:10.719950 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:10.723647 (+ 3697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:10.723651 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:10.723654 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:10.724017 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:10.724406 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:10.724409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:10.724411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:10.724538 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:10.725131 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:10.725134 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:10.725266 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 03:17:10.725416 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 03:17:10.725619 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 03:17:10.726109 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.712327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.11.60.61:32865)
0811 03:17:10.712542 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:10.712551 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:10.712578 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:17:10.712818 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:10.712827 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:10.712912 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:10.713036 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:10.713883 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.714712 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.716010 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:10.716465 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:10.716603 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:10.720156 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:10.720163 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:10.720168 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:10.720213 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:10.723505 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:10.724137 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:10.724143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:10.724146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:10.724294 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:10.724640 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:10.724645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:10.724648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:10.725017 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:10.725167 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:10.725553 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 03:17:10.725788 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 03:17:10.726089 4101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } attempt: 2
I20260811 03:17:10.727164 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.726680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55894)
0811 03:17:10.726884 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.727009 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 03:17:10.727840 4022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:17:10.728292 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.727883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55898)
0811 03:17:10.728064 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.728159 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 03:17:10.728850 4101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:36527: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
I20260811 03:17:10.728971 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.728529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55904)
0811 03:17:10.728685 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.728777 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 03:17:10.729825 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.729509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55920)
0811 03:17:10.729652 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.729724 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 03:17:10.730093 4022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:36527: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
I20260811 03:17:10.781330 4022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } attempt: 1
I20260811 03:17:10.782672 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.782122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55930)
0811 03:17:10.782394 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.782505 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 03:17:10.783758 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.783463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55938)
0811 03:17:10.783595 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.783663 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 03:17:10.784042 4022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:36527: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
I20260811 03:17:10.820124 4101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } attempt: 3
I20260811 03:17:10.821586 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.820928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55946)
0811 03:17:10.821210 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.821357 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 03:17:10.822808 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.822275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55958)
0811 03:17:10.822567 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.822665 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
W20260811 03:17:10.823026 4101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:36527: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
I20260811 03:17:10.852284 4022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } attempt: 2
I20260811 03:17:10.853624 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.853125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55972)
0811 03:17:10.853351 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.853461 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 03:17:10.854614 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.854255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55978)
0811 03:17:10.854433 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.854506 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 03:17:10.854827 4022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:36527: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
I20260811 03:17:10.961042 4022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } attempt: 3
I20260811 03:17:10.962213 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.961714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55980)
0811 03:17:10.961924 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.962042 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 03:17:10.963268 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.962893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:55986)
0811 03:17:10.963101 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.963165 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260811 03:17:10.963464 4022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:36527: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
I20260811 03:17:10.966256 4101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } attempt: 4
I20260811 03:17:10.967368 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.966841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:56002)
0811 03:17:10.967081 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.967185 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 03:17:10.968423 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:10.968012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:56016)
0811 03:17:10.968169 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:10.968267 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 03:17:10.968641 4101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:36527: Network error: Client connection negotiation failed: client connection to 127.11.60.60:36527: connect: Connection refused (error 111)
W20260811 03:17:10.982892 4109 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:17:10.983245 4109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:17:10.983351 4109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:17:10.983489 4109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:17:10.990948 4109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:17:10.991030 4109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:17:10.991156 4109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:17:10.991236 4109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:17:10.991356 4109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:17:11.001173 4109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:39189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:39905,127.11.60.61:32865,127.11.60.60:36527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab
--principal=kudu/127.11.60.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.60:36527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.60.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:17:11.002662 4109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:17:11.004009 4109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:17:11.016398 4114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:11.016512 4117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:11.016513 4115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:17:11.017117 4109 server_base.cc:1034] running on GCE node
I20260811 03:17:11.017835 4109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:17:11.019259 4109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:17:11.020519 4109 hybrid_clock.cc:648] HybridClock initialized: now 1786418231020446 us; error 37 us; skew 500 ppm
I20260811 03:17:11.020848 4109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:17:11 dist-test-slave-tm1g krb5kdc[3939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418231, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:17:11.028550 4109 init.cc:377] Logged in from keytab as kudu/127.11.60.60@KRBTEST.COM (short username kudu)
I20260811 03:17:11.030026 4109 webserver.cc:460] Webserver started at http://127.11.60.60:44321/ using document root <none> and password file <none>
I20260811 03:17:11.030782 4109 fs_manager.cc:359] Metadata directory not provided
I20260811 03:17:11.030872 4109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:17:11.031189 4109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:17:11.034159 4109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/data/instance:
uuid: "21692663a9034c279e64e585cdceaf7f"
format_stamp: "Formatted at 2026-08-11 03:17:11 on dist-test-slave-tm1g"
server_key: "c87e0c388bc6e9fabd5a001d2ab3f2e1"
server_key_iv: "c95e04581d914cafba81ca26d8237f96"
server_key_version: "encryptionkey@0"
I20260811 03:17:11.035055 4109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/wal/instance:
uuid: "21692663a9034c279e64e585cdceaf7f"
format_stamp: "Formatted at 2026-08-11 03:17:11 on dist-test-slave-tm1g"
server_key: "c87e0c388bc6e9fabd5a001d2ab3f2e1"
server_key_iv: "c95e04581d914cafba81ca26d8237f96"
server_key_version: "encryptionkey@0"
I20260811 03:17:11.040161 4109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:17:11.043615 4125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:17:11.044842 4109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:17:11.045006 4109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/wal
uuid: "21692663a9034c279e64e585cdceaf7f"
format_stamp: "Formatted at 2026-08-11 03:17:11 on dist-test-slave-tm1g"
server_key: "c87e0c388bc6e9fabd5a001d2ab3f2e1"
server_key_iv: "c95e04581d914cafba81ca26d8237f96"
server_key_version: "encryptionkey@0"
I20260811 03:17:11.045233 4109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:17:11.059585 4109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:17:11.063584 4109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:17:11.063864 4109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:17:11.083213 4109 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.60:36527
I20260811 03:17:11.083225 4178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.60:36527 every 8 connection(s)
I20260811 03:17:11.084609 4109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
I20260811 03:17:11.088366 4179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:17:11.092074 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 4109
I20260811 03:17:11.092274 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417858274425-11504-0/minicluster-data/master-2/wal/instance
I20260811 03:17:11.092824 11504 external_mini_cluster.cc:1428] Setting key e2542612a1ecc3d097702a370099d8cb
I20260811 03:17:11.096448 4179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:17:11 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
Aug 11 03:17:11 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418231, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:17:11.116133 4184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.100329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.11.60.62:39905)
0811 03:17:11.100838 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.100846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.101342 (+ 496us) server_negotiation.cc:407] Connection header received
0811 03:17:11.101571 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.101586 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.101657 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.101775 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.103003 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.103701 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.105302 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.105737 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.105874 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.109745 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.109750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.109754 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.109795 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.113007 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.113728 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.113734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.113738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.113881 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.114276 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.114284 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.114288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.114612 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.114771 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.115465 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.115706 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":273,"threads_started":1}
I20260811 03:17:11.116590 4185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.100290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:39905 (local address 127.0.0.1:59376)
0811 03:17:11.101192 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:11.101236 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:11.101419 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:11.101844 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:11.101851 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:11.102576 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:11.102841 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:11.102850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.103865 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.103872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:11.105136 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:11.105149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.105968 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.105976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:11.106139 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.106816 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:11.106838 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:11.109540 (+ 2702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:11.113136 (+ 3596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:11.113150 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:11.113157 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:11.113604 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:11.114006 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:11.114011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:11.114021 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:11.114162 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:11.114746 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:11.114755 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:11.114954 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 03:17:11.115141 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 03:17:11.115595 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":92,"thread_start_us":539,"threads_started":1}
I20260811 03:17:11.116607 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.094872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56030 (local address 127.11.60.60:36527)
0811 03:17:11.095681 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.095692 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.095745 (+ 53us) server_negotiation.cc:407] Connection header received
0811 03:17:11.097366 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.097390 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.097784 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.098154 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.100088 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.101976 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.103393 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.104057 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.104281 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.108869 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.108877 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.108892 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.108923 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.112103 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.112996 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.113003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.113009 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.113164 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.113633 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.113640 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.113648 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.114159 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.114521 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.115025 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.115426 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"spinlock_wait_cycles":20864,"thread_start_us":393,"threads_started":1}
I20260811 03:17:11.119024 4179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:17:11 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418231, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
Aug 11 03:17:11 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:17:11.133980 4185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.119684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:32865 (local address 127.0.0.1:58286)
0811 03:17:11.119991 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:11.120009 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:11.120099 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:11.120453 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:11.120458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:11.120951 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:11.121143 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:11.121158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.122402 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.122406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:11.123761 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:11.123776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.124540 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.124545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:11.124685 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.125319 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:11.125342 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:11.127802 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:11.131397 (+ 3595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:11.131401 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:11.131403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:11.131774 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:11.132218 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:11.132222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:11.132224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:11.132364 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:11.132986 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:11.132992 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:11.133162 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 03:17:11.133338 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 03:17:11.133611 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260811 03:17:11.133996 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.119947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58286 (local address 127.11.60.61:32865)
0811 03:17:11.120131 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.120137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.120159 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:17:11.120223 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.120228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.120305 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.120409 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.121310 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.122183 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.123912 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.124400 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.124520 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.128006 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.128011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.128014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.128046 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.131268 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.131895 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.131904 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.131908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.132090 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.132476 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.132482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.132486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.132848 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.132981 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.133415 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.133661 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 03:17:11.135270 4179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:17:11.135888 4184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.119780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59380 (local address 127.11.60.62:39905)
0811 03:17:11.120151 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.120157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.120175 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:17:11.120229 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.120233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.120305 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.120407 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.121815 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.122697 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.124319 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.124948 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.125079 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.129498 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.129510 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.129514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.129544 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.132724 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.133519 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.133524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.133528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.133666 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.134337 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.134346 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.134352 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.134715 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.135003 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.135359 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.135593 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264}
I20260811 03:17:11.140703 4022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } attempt: 4
Aug 11 03:17:11 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418231, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
Aug 11 03:17:11 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
Aug 11 03:17:11 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
I20260811 03:17:11.151785 4185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.136174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:56032)
0811 03:17:11.136371 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:11.136392 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:11.136509 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:11.137019 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:11.137023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:11.137425 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:11.137622 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:11.137634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.138887 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.138895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:11.140036 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:11.140049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.140820 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.140825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:11.141065 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.141827 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:11.141855 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:11.144459 (+ 2604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:11.148789 (+ 4330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:11.148796 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:11.148800 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:11.149491 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:11.149958 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:11.149966 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:11.149972 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:11.150146 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:11.150694 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:11.150698 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:11.150838 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 03:17:11.151016 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 03:17:11.151322 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 03:17:11.151825 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.136199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.11.60.60:36527)
0811 03:17:11.136354 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.136360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.136448 (+ 88us) server_negotiation.cc:407] Connection header received
0811 03:17:11.136678 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.136683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.136762 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.136867 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.137786 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.138738 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.140170 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.140676 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.140813 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.144640 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.144644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.144648 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.144678 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.148662 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.149620 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.149626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.149629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.149810 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.150253 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.150258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.150261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.150579 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.150731 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.151101 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.151512 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":147,"server-negotiator.queue_time_us":53}
I20260811 03:17:11.155570 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.137593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58298 (local address 127.11.60.61:32865)
0811 03:17:11.137789 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.137794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.137811 (+ 17us) server_negotiation.cc:407] Connection header received
0811 03:17:11.137940 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.137947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.138016 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.138115 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.139518 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.140291 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.142234 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.142928 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.143057 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.148263 (+ 5206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.148270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.148274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.148337 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.152288 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.153205 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.153210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.153213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.153393 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.153877 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.153882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.153886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.154261 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.154579 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.154861 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.155108 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 03:17:11.158156 4179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f: Bootstrap starting.
I20260811 03:17:11.158576 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.141294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:56048)
0811 03:17:11.141479 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:11.141497 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:11.141585 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:11.142564 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:11.142569 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:11.143053 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:11.143277 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:11.143292 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.144934 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.144940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:11.146482 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:11.146495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.147221 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.147228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:11.147450 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.148033 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:11.148058 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:11.150926 (+ 2868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:11.155404 (+ 4478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:11.155409 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:11.155412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:11.156080 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:11.156558 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:11.156563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:11.156567 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:11.156717 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:11.157500 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:11.157506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:11.157707 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 03:17:11.157837 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 03:17:11.158157 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 03:17:11.159242 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.141383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.11.60.60:36527)
0811 03:17:11.142215 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.142221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.142247 (+ 26us) server_negotiation.cc:407] Connection header received
0811 03:17:11.142333 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.142337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.142402 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.142517 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.143454 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.144765 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.146639 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.147057 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.147171 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.151211 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.151218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.151222 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.151258 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.155239 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.156257 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.156262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.156265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.156425 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.156952 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.156955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.156957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.157329 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.158509 (+ 1180us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.158630 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.158847 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":422,"threads_started":1}
I20260811 03:17:11.161141 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:17:11.162755 4179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f: Neither blocks nor log segments found. Creating new log.
I20260811 03:17:11.163666 4022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009: Bootstrap starting.
I20260811 03:17:11.164136 4179 log.cc:824] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f: Log is configured to *not* fsync() on all Append() calls
I20260811 03:17:11.167106 4179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f: No bootstrap required, opened a new log
I20260811 03:17:11.167874 4022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009: Neither blocks nor log segments found. Creating new log.
I20260811 03:17:11.164772 3996 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cbb0ad4265e049d187fdec63477e9009"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59380
I20260811 03:17:11.169212 4022 log.cc:824] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009: Log is configured to *not* fsync() on all Append() calls
I20260811 03:17:11.172506 4022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009: No bootstrap required, opened a new log
I20260811 03:17:11.177150 4179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } }
I20260811 03:17:11.177593 4179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21692663a9034c279e64e585cdceaf7f, State: Initialized, Role: FOLLOWER
I20260811 03:17:11.178121 4179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } }
I20260811 03:17:11.179116 4189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } } }
I20260811 03:17:11.179553 4189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:11.180433 4179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:17:11.180112 4022 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } }
I20260811 03:17:11.180578 4022 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbb0ad4265e049d187fdec63477e9009, State: Initialized, Role: FOLLOWER
I20260811 03:17:11.181069 4022 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } }
I20260811 03:17:11.182140 4190 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } } }
I20260811 03:17:11.182650 4190 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:11.183594 4022 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:17:11.183652 3996 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 03:17:11.183858 3996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:17:11.187219 3996 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } }
I20260811 03:17:11.189704 4154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbb0ad4265e049d187fdec63477e9009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "21692663a9034c279e64e585cdceaf7f"
I20260811 03:17:11.190155 4154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:17:11.193634 4154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbb0ad4265e049d187fdec63477e9009 in term 1.
I20260811 03:17:11.194365 3996 leader_election.cc:288] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [CANDIDATE]: Term 1 election: Requested vote from peers e8dd2fbea7ba4671b9371c1e5855c6f6 (127.11.60.61:32865), 21692663a9034c279e64e585cdceaf7f (127.11.60.60:36527)
I20260811 03:17:11.194360 3971 leader_election.cc:302] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21692663a9034c279e64e585cdceaf7f, cbb0ad4265e049d187fdec63477e9009; no voters:
I20260811 03:17:11.194947 4190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:17:11.206476 4190 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [term 1 LEADER]: Becoming Leader. State: Replica: cbb0ad4265e049d187fdec63477e9009, State: Running, Role: LEADER
I20260811 03:17:11.207880 4190 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } }
I20260811 03:17:11.218456 4220 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 03:17:11.222052 4109 master_runner.cc:186] Error getting master registration for 127.11.60.62:39905: OK, instance_id { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" instance_seqno: 1786418229999399 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:17:11.236923 4109 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 03:17:11.238626 3974 tablet.cc:2369] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:17:11.239785 4207 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb0ad4265e049d187fdec63477e9009. Latest consensus state: current_term: 1 leader_uuid: "cbb0ad4265e049d187fdec63477e9009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } } }
I20260811 03:17:11.240227 4207 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:11.251526 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.222838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58302 (local address 127.11.60.61:32865)
0811 03:17:11.223029 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.223035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.229874 (+ 6839us) server_negotiation.cc:407] Connection header received
0811 03:17:11.230279 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.230287 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.230414 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.230492 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.232815 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.233902 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.235976 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.236693 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.236796 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.241933 (+ 5137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.241940 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.241944 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.241991 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.247417 (+ 5426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.248383 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.248437 (+ 54us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.248440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.248617 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.249062 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.249069 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.249074 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.249533 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.249800 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.250838 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.251106 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 03:17:11.252554 4184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.221163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.11.60.62:39905)
0811 03:17:11.221364 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.221370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.222877 (+ 1507us) server_negotiation.cc:407] Connection header received
0811 03:17:11.223314 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.223322 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.223430 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.223523 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.228810 (+ 5287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.230135 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.233613 (+ 3478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.234367 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.234470 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.241995 (+ 7525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.242002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.242007 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.242046 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.247504 (+ 5458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.248730 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.248736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.248739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.248919 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.249691 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.249696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.249713 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.250087 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.250562 (+ 475us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.251499 (+ 937us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.251900 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 03:17:11.254428 4226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:17:11.262617 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.221940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56054 (local address 127.11.60.60:36527)
0811 03:17:11.222117 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.222124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.244220 (+ 22096us) server_negotiation.cc:407] Connection header received
0811 03:17:11.244405 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.244413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.244526 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.244631 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.246678 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.248130 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.250978 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.251885 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.252022 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.255879 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.255885 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.255897 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.255946 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.259418 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.260251 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.260256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.260260 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.260404 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.260819 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.260824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.260827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.261121 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.261370 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.261786 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.262228 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 03:17:11.267905 4101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } attempt: 5
I20260811 03:17:11.259449 3952 master_runner.cc:186] Error getting master registration for 127.11.60.61:32865: OK, instance_id { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" instance_seqno: 1786418230542100 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:17:11.274286 3952 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 03:17:11.274523 4226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 03:17:11 dist-test-slave-tm1g krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418230, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
I20260811 03:17:11.286216 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.268560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56060 (local address 127.11.60.60:36527)
0811 03:17:11.269130 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.269160 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.269180 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:17:11.269249 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.269254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.269385 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.270788 (+ 1403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.270983 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.272185 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.274036 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.275361 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.275490 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.279373 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:11.279380 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:11.279384 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:11.279426 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:11.283156 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.284035 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.284040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.284043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.284190 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:11.284583 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:11.284588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:11.284593 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:11.284939 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:11.285102 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:11.285645 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 03:17:11.285849 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401}
I20260811 03:17:11.286293 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:11.268493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:36527 (local address 127.0.0.1:56060)
0811 03:17:11.268736 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:11.268755 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:11.268855 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:11.269617 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:11.269624 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:11.270184 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:11.270433 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:11.270447 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.272305 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.272312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:11.273791 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:11.273806 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:11.275520 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:11.275527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:11.275753 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:11.276400 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:11.276423 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:11.279154 (+ 2731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:11.283306 (+ 4152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:11.283313 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:11.283318 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:11.283868 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:11.284305 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:11.284310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:11.284314 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:11.284469 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:11.285115 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:11.285142 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:11.285363 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 03:17:11.285518 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 03:17:11.285832 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 03:17:11.290486 4154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [term 1 FOLLOWER]: Refusing update from remote peer cbb0ad4265e049d187fdec63477e9009: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:17:11.291453 4207 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:17:11.291978 4101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6: Bootstrap starting.
I20260811 03:17:11.297175 4101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6: Neither blocks nor log segments found. Creating new log.
I20260811 03:17:11.298538 4101 log.cc:824] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6: Log is configured to *not* fsync() on all Append() calls
I20260811 03:17:11.307504 4101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6: No bootstrap required, opened a new log
I20260811 03:17:11.311993 4189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb0ad4265e049d187fdec63477e9009. Latest consensus state: current_term: 1 leader_uuid: "cbb0ad4265e049d187fdec63477e9009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } } }
I20260811 03:17:11.312286 4189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:11.313629 4229 mvcc.cc:205] Tried to move back new op lower bound from 7317169075355316224 to 7317169075042459648. Current Snapshot: MvccSnapshot[applied={T|T < 7317169075355316224}]
I20260811 03:17:11.315781 4207 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cbb0ad4265e049d187fdec63477e9009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } } }
I20260811 03:17:11.316051 4207 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:11.318634 4226 catalog_manager.cc:1347] Generated new cluster ID: 9487403d46de408492352190bc4cd335
I20260811 03:17:11.318743 4226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:17:11.318967 4101 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } }
I20260811 03:17:11.319451 4101 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8dd2fbea7ba4671b9371c1e5855c6f6, State: Initialized, Role: FOLLOWER
I20260811 03:17:11.320050 4101 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } }
I20260811 03:17:11.327077 4235 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } } }
I20260811 03:17:11.329386 4235 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:11.329254 4101 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:17:11.329661 4074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:17:11.330403 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbb0ad4265e049d187fdec63477e9009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6"
I20260811 03:17:11.335747 4074 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [term 1 FOLLOWER]: Refusing update from remote peer cbb0ad4265e049d187fdec63477e9009: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:17:11.340466 4075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbb0ad4265e049d187fdec63477e9009 in term 1.
I20260811 03:17:11.342943 4189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cbb0ad4265e049d187fdec63477e9009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } } }
I20260811 03:17:11.343235 4189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:11.347285 4190 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:17:11.386185 4235 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb0ad4265e049d187fdec63477e9009. Latest consensus state: current_term: 1 leader_uuid: "cbb0ad4265e049d187fdec63477e9009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } } }
I20260811 03:17:11.386531 4235 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:11.386833 4245 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cbb0ad4265e049d187fdec63477e9009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } } }
I20260811 03:17:11.387140 4245 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:11.388271 4251 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 03:17:11.410907 4190 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cbb0ad4265e049d187fdec63477e9009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb0ad4265e049d187fdec63477e9009" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 39905 } } peers { permanent_uuid: "e8dd2fbea7ba4671b9371c1e5855c6f6" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 32865 } } peers { permanent_uuid: "21692663a9034c279e64e585cdceaf7f" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 36527 } } }
I20260811 03:17:11.411263 4190 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:11.413197 4226 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:17:11.415134 4226 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:17:11.438199 4226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbb0ad4265e049d187fdec63477e9009: Generated new TSK 0
I20260811 03:17:11.439448 4226 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 03:17:11.520313 4184 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.516726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.11.60.62:39905)
0811 03:17:11.517005 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.517015 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.517868 (+ 853us) server_negotiation.cc:407] Connection header received
0811 03:17:11.518206 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.518213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.518325 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.518413 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.519946 (+ 1533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59408: BlockingRecv error: recv got EOF from 127.0.0.1:59408 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 03:17:11.523017 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.518102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.11.60.61:32865)
0811 03:17:11.518288 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.518294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.521246 (+ 2952us) server_negotiation.cc:407] Connection header received
0811 03:17:11.521515 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.521522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.521627 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.521739 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.522772 (+ 1033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58314: BlockingRecv error: recv got EOF from 127.0.0.1:58314 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 03:17:11.523360 4186 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.517322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56062 (local address 127.11.60.60:36527)
0811 03:17:11.517508 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.517516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.521248 (+ 3732us) server_negotiation.cc:407] Connection header received
0811 03:17:11.521736 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.521742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.521817 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.521926 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.523115 (+ 1189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56062: BlockingRecv error: recv got EOF from 127.0.0.1:56062 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 03:17:11.530417 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.528426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58322 (local address 127.11.60.61:32865)
0811 03:17:11.528650 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.528655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.528948 (+ 293us) server_negotiation.cc:407] Connection header received
0811 03:17:11.529150 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.529154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.529243 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.529364 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.530123 (+ 759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58322: BlockingRecv error: recv got EOF from 127.0.0.1:58322 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 03:17:11.536422 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.535212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.11.60.61:32865)
0811 03:17:11.535379 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.535385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.535429 (+ 44us) server_negotiation.cc:407] Connection header received
0811 03:17:11.535485 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.535489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.535559 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.535647 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.536258 (+ 611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58336: BlockingRecv error: recv got EOF from 127.0.0.1:58336 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 03:17:11.543551 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.541477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.11.60.61:32865)
0811 03:17:11.541635 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.541641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.542123 (+ 482us) server_negotiation.cc:407] Connection header received
0811 03:17:11.542291 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.542295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.542428 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.542512 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.543309 (+ 797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58344: BlockingRecv error: recv got EOF from 127.0.0.1:58344 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 03:17:11.562436 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.560174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.11.60.61:32865)
0811 03:17:11.560343 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.560349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.560896 (+ 547us) server_negotiation.cc:407] Connection header received
0811 03:17:11.561077 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.561083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.561196 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.561315 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.562194 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58356: BlockingRecv error: recv got EOF from 127.0.0.1:58356 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 03:17:11.575965 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.573941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.11.60.61:32865)
0811 03:17:11.574185 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.574192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.574579 (+ 387us) server_negotiation.cc:407] Connection header received
0811 03:17:11.574762 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.574768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.574877 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.575004 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.575705 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58372: BlockingRecv error: recv got EOF from 127.0.0.1:58372 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 03:17:11.597849 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.595663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.11.60.61:32865)
0811 03:17:11.595905 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.595911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.596299 (+ 388us) server_negotiation.cc:407] Connection header received
0811 03:17:11.596502 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.596505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.596594 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.596684 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.597607 (+ 923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58378: BlockingRecv error: recv got EOF from 127.0.0.1:58378 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 03:17:11.636307 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.634141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.11.60.61:32865)
0811 03:17:11.634400 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.634407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.634835 (+ 428us) server_negotiation.cc:407] Connection header received
0811 03:17:11.634990 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.634996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.635097 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.635240 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.636012 (+ 772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58382: BlockingRecv error: recv got EOF from 127.0.0.1:58382 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 03:17:11.706239 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.703794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.11.60.61:32865)
0811 03:17:11.704022 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.704028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.704426 (+ 398us) server_negotiation.cc:407] Connection header received
0811 03:17:11.704601 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.704607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.704721 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.704858 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.706010 (+ 1152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58386: BlockingRecv error: recv got EOF from 127.0.0.1:58386 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 03:17:11.841482 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:11.839152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.11.60.61:32865)
0811 03:17:11.839359 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:11.839366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:11.839964 (+ 598us) server_negotiation.cc:407] Connection header received
0811 03:17:11.840207 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:11.840217 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:11.840329 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:11.840448 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:11.841234 (+ 786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58398: BlockingRecv error: recv got EOF from 127.0.0.1:58398 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 03:17:12.103255 4106 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:12.101541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58404 (local address 127.11.60.61:32865)
0811 03:17:12.101728 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:12.101733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:12.101994 (+ 261us) server_negotiation.cc:407] Connection header received
0811 03:17:12.102157 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:12.102161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:12.102233 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:12.102397 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:12.103090 (+ 693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58404: BlockingRecv error: recv got EOF from 127.0.0.1:58404 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 03:17:12.222957 4220 catalog_manager.cc:1259] Loaded cluster ID: 9487403d46de408492352190bc4cd335
I20260811 03:17:12.223075 4220 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f: loading cluster ID for follower catalog manager: success
I20260811 03:17:12.225798 4220 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f: acquiring CA information for follower catalog manager: success
I20260811 03:17:12.227730 4220 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 21692663a9034c279e64e585cdceaf7f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:17:12.393419 4251 catalog_manager.cc:1259] Loaded cluster ID: 9487403d46de408492352190bc4cd335
I20260811 03:17:12.393580 4251 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6: loading cluster ID for follower catalog manager: success
I20260811 03:17:12.397451 4251 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6: acquiring CA information for follower catalog manager: success
I20260811 03:17:12.400106 4251 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8dd2fbea7ba4671b9371c1e5855c6f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:17:12.626425 4326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:12.619078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58418 (local address 127.11.60.61:32865)
0811 03:17:12.619911 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:12.619919 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:12.619937 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:17:12.620024 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:12.620028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:12.620180 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:12.620284 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:17:12.620858 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:12.621728 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:12.623843 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:12.624295 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:12.624390 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:12.625053 (+ 663us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:17:12.625642 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:17:12.625755 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:12.625943 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 03:17:12.626016 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":520,"threads_started":1}
I20260811 03:17:12.643395 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:12.634835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56076 (local address 127.11.60.60:36527)
0811 03:17:12.637062 (+ 2227us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:12.637067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:12.637565 (+ 498us) server_negotiation.cc:407] Connection header received
0811 03:17:12.637836 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:12.637840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:12.638004 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:12.638199 (+ 195us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:17:12.638681 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:12.639482 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:12.640980 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:12.641417 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:12.641556 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:12.642081 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:17:12.642736 (+ 655us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:17:12.642897 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:12.642965 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 03:17:12.643059 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2102,"thread_start_us":1937,"threads_started":1}
I20260811 03:17:12.671999 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:12.650680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59420 (local address 127.11.60.62:39905)
0811 03:17:12.662795 (+ 12115us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:12.662801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:12.663319 (+ 518us) server_negotiation.cc:407] Connection header received
0811 03:17:12.664110 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:12.664119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:12.664365 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:12.664594 (+ 229us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:17:12.665373 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:12.666583 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:12.668814 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:12.669481 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:12.669624 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:12.670234 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:17:12.670827 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:17:12.670957 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:12.671501 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 03:17:12.671616 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11937,"thread_start_us":11797,"threads_started":1}
I20260811 03:17:12.672869 4326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:12.663515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58430 (local address 127.11.60.61:32865)
0811 03:17:12.663743 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:12.663749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:12.666909 (+ 3160us) server_negotiation.cc:407] Connection header received
0811 03:17:12.667102 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:12.667107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:12.667191 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:12.667289 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:17:12.667873 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:12.668609 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:12.670266 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:12.671058 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:12.671219 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:12.671985 (+ 766us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:17:12.672115 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:17:12.672184 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:12.672441 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 03:17:12.672553 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 03:17:12.677242 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:12.650837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56082 (local address 127.11.60.60:36527)
0811 03:17:12.656260 (+ 5423us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:12.656269 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:12.663360 (+ 7091us) server_negotiation.cc:407] Connection header received
0811 03:17:12.663625 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:12.663630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:12.663723 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:12.663816 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:17:12.671040 (+ 7224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:12.672135 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:12.674022 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:12.674598 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:12.674754 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:12.676454 (+ 1700us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:17:12.676589 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:17:12.676694 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:12.676880 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 03:17:12.676976 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5237}
I20260811 03:17:12.677968 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 3952
I20260811 03:17:12.679111 4017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:17:12.813705 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 3952
I20260811 03:17:12.839120 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 4030
I20260811 03:17:12.840370 4095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:17:12.951467 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 4030
I20260811 03:17:12.974895 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 4109
I20260811 03:17:12.976078 4174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:17:13.085831 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 4109
2026-08-11T03:17:13Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5054 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:17:13 dist-test-slave-tm1g krb5kdc[4346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:17:13 dist-test-slave-tm1g krb5kdc[4346](info): set up 2 sockets
Aug 11 03:17:13 dist-test-slave-tm1g krb5kdc[4346](info): commencing operation
krb5kdc: starting...
W20260811 03:17:14.296263 11504 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/test-user.keytab.
Aug 11 03:17:14 dist-test-slave-tm1g krb5kdc[4346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:17:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:17:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.60.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
Entry for principal kudu/127.11.60.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab.
I20260811 03:17:14.633976 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40183
--webserver_interface=127.11.60.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:40183,127.11.60.61:42925,127.11.60.60:44551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:17:14.983114 4361 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:17:14.983466 4361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:17:14.983623 4361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:17:14.983753 4361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:17:14.991063 4361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:17:14.991149 4361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:17:14.991250 4361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:17:14.991339 4361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:17:14.991458 4361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:17:15.001230 4361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:40183,127.11.60.61:42925,127.11.60.60:44551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.62.keytab
--principal=kudu/127.11.60.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.62:40183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.60.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:17:15.002625 4361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:17:15.004011 4361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:17:15.016561 4369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:15.016515 4367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:15.016606 4366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:17:15.017194 4361 server_base.cc:1034] running on GCE node
I20260811 03:17:15.018102 4361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:17:15.019610 4361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:17:15.020836 4361 hybrid_clock.cc:648] HybridClock initialized: now 1786418235020765 us; error 41 us; skew 500 ppm
I20260811 03:17:15.021139 4361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:17:15 dist-test-slave-tm1g krb5kdc[4346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418235, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:17:15.024314 4361 init.cc:377] Logged in from keytab as kudu/127.11.60.62@KRBTEST.COM (short username kudu)
I20260811 03:17:15.025791 4361 webserver.cc:460] Webserver started at http://127.11.60.62:37829/ using document root <none> and password file <none>
I20260811 03:17:15.026619 4361 fs_manager.cc:359] Metadata directory not provided
I20260811 03:17:15.026757 4361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:17:15.027103 4361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:17:15.030201 4361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/data/instance:
uuid: "027b881a8617460f951497682be3d670"
format_stamp: "Formatted at 2026-08-11 03:17:15 on dist-test-slave-tm1g"
server_key: "25f0aacb728cb2b320bb9ecfbd48ba67"
server_key_iv: "4291bb8a03be2a55506cefd5cf950e3e"
server_key_version: "encryptionkey@0"
I20260811 03:17:15.031111 4361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/wal/instance:
uuid: "027b881a8617460f951497682be3d670"
format_stamp: "Formatted at 2026-08-11 03:17:15 on dist-test-slave-tm1g"
server_key: "25f0aacb728cb2b320bb9ecfbd48ba67"
server_key_iv: "4291bb8a03be2a55506cefd5cf950e3e"
server_key_version: "encryptionkey@0"
I20260811 03:17:15.036319 4361 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 03:17:15.039800 4377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:17:15.041049 4361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:17:15.041235 4361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/wal
uuid: "027b881a8617460f951497682be3d670"
format_stamp: "Formatted at 2026-08-11 03:17:15 on dist-test-slave-tm1g"
server_key: "25f0aacb728cb2b320bb9ecfbd48ba67"
server_key_iv: "4291bb8a03be2a55506cefd5cf950e3e"
server_key_version: "encryptionkey@0"
I20260811 03:17:15.041481 4361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:17:15.078414 4361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:17:15.082509 4361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:17:15.082803 4361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:17:15.102166 4361 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.62:40183
I20260811 03:17:15.102190 4432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.62:40183 every 8 connection(s)
I20260811 03:17:15.103590 4361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/data/info.pb
I20260811 03:17:15.106153 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 4361
I20260811 03:17:15.106351 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-0/wal/instance
I20260811 03:17:15.106823 11504 external_mini_cluster.cc:1428] Setting key 0fda80e158a698990a91b4e59762904d
I20260811 03:17:15.107574 4433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:17:15.114032 4433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:17:15 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418235, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:17:15.136452 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.117319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40183 (local address 127.0.0.1:54318)
0811 03:17:15.118092 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.118130 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:15.120283 (+ 2153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:15.121103 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:15.121110 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:15.121803 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:15.122134 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:15.122151 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.123752 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.123758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:15.124833 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:15.124845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.125793 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.125800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:15.126010 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:15.126730 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:15.126752 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:15.128910 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:15.132231 (+ 3321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:15.132245 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:15.132256 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:15.132645 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:15.133052 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:15.133058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:15.133061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:15.133233 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:15.133986 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:15.133993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:15.134224 (+ 231us) client_negotiation.cc:769] Sending connection context
0811 03:17:15.134532 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 03:17:15.135115 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":136,"thread_start_us":333,"threads_started":1}
I20260811 03:17:15.136487 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.118604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54318 (local address 127.11.60.62:40183)
0811 03:17:15.119353 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:15.119363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:15.119431 (+ 68us) server_negotiation.cc:407] Connection header received
0811 03:17:15.120649 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:15.120655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:15.120884 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:15.121098 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:15.122379 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.123594 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.124978 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.125557 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.125761 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:15.129154 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:15.129162 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:15.129173 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:15.129201 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:15.132097 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:15.132780 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:15.132788 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:15.132794 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:15.132941 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:15.133353 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:15.133359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:15.133367 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:15.133847 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:15.134080 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:15.134656 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 03:17:15.134993 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":360,"threads_started":1}
WARNING: no policy specified for kudu/127.11.60.61@KRBTEST.COM; defaulting to no policy
I20260811 03:17:15.139750 4433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.61@KRBTEST.COM" created.
I20260811 03:17:15.140996 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.140599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:36972)
0811 03:17:15.140774 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.140883 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 03:17:15.141304 4381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
I20260811 03:17:15.143000 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.142612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:36982)
0811 03:17:15.142810 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.142878 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 03:17:15.143199 4433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:42925: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab.
Entry for principal kudu/127.11.60.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab.
I20260811 03:17:15.178980 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.61:42925
--webserver_interface=127.11.60.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab
--principal=kudu/127.11.60.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:40183,127.11.60.61:42925,127.11.60.60:44551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:17:15.192549 4433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } attempt: 1
I20260811 03:17:15.193925 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.193332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:36996)
0811 03:17:15.193593 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.193724 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 03:17:15.195142 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.194685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:37012)
0811 03:17:15.194868 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.194960 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 03:17:15.195363 4433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:42925: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
I20260811 03:17:15.263652 4433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } attempt: 2
I20260811 03:17:15.265115 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.264424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:37016)
0811 03:17:15.264743 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.264898 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 03:17:15.266374 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.265821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:37032)
0811 03:17:15.266025 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.266147 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 03:17:15.266812 4433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:42925: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
I20260811 03:17:15.358146 4433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } attempt: 3
I20260811 03:17:15.359467 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.358933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:37042)
0811 03:17:15.359173 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.359290 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 03:17:15.360611 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.360160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:37058)
0811 03:17:15.360319 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.360414 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 03:17:15.360884 4433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:42925: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
I20260811 03:17:15.504205 4433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } attempt: 4
I20260811 03:17:15.505702 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.504968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:37060)
0811 03:17:15.505415 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.505528 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238}
I20260811 03:17:15.506951 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.506507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:37076)
0811 03:17:15.506671 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.506783 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 03:17:15.507217 4433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.61:42925: Network error: Client connection negotiation failed: client connection to 127.11.60.61:42925: connect: Connection refused (error 111)
W20260811 03:17:15.528980 4440 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:17:15.529408 4440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:17:15.529521 4440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:17:15.529645 4440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:17:15.537346 4440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:17:15.537438 4440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:17:15.537544 4440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:17:15.537639 4440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:17:15.537752 4440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:17:15.548050 4440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:40183,127.11.60.61:42925,127.11.60.60:44551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.61.keytab
--principal=kudu/127.11.60.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.61:42925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.60.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:17:15.549413 4440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:17:15.550833 4440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:17:15.563427 4449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:15.563627 4445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:15.565586 4447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:17:15.566141 4440 server_base.cc:1034] running on GCE node
I20260811 03:17:15.567103 4440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:17:15.568691 4440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:17:15.569963 4440 hybrid_clock.cc:648] HybridClock initialized: now 1786418235569891 us; error 43 us; skew 500 ppm
I20260811 03:17:15.570379 4440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:17:15 dist-test-slave-tm1g krb5kdc[4346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418235, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:17:15.573752 4440 init.cc:377] Logged in from keytab as kudu/127.11.60.61@KRBTEST.COM (short username kudu)
I20260811 03:17:15.575326 4440 webserver.cc:460] Webserver started at http://127.11.60.61:42061/ using document root <none> and password file <none>
I20260811 03:17:15.576135 4440 fs_manager.cc:359] Metadata directory not provided
I20260811 03:17:15.576272 4440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:17:15.576608 4440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:17:15.579775 4440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/data/instance:
uuid: "6614417d01f54471aa6bc38c61328923"
format_stamp: "Formatted at 2026-08-11 03:17:15 on dist-test-slave-tm1g"
server_key: "eff4828be335963037634e1b89caa079"
server_key_iv: "4a2dd5a1769dd042ba8cbbd99a13f507"
server_key_version: "encryptionkey@0"
I20260811 03:17:15.580605 4440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/wal/instance:
uuid: "6614417d01f54471aa6bc38c61328923"
format_stamp: "Formatted at 2026-08-11 03:17:15 on dist-test-slave-tm1g"
server_key: "eff4828be335963037634e1b89caa079"
server_key_iv: "4a2dd5a1769dd042ba8cbbd99a13f507"
server_key_version: "encryptionkey@0"
I20260811 03:17:15.586145 4440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:17:15.589870 4456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:17:15.591372 4440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:17:15.591562 4440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/wal
uuid: "6614417d01f54471aa6bc38c61328923"
format_stamp: "Formatted at 2026-08-11 03:17:15 on dist-test-slave-tm1g"
server_key: "eff4828be335963037634e1b89caa079"
server_key_iv: "4a2dd5a1769dd042ba8cbbd99a13f507"
server_key_version: "encryptionkey@0"
I20260811 03:17:15.591784 4440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:17:15.609812 4440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:17:15.614442 4440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:17:15.614719 4440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:17:15.633776 4440 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.61:42925
I20260811 03:17:15.633800 4507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.61:42925 every 8 connection(s)
I20260811 03:17:15.635217 4440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/data/info.pb
I20260811 03:17:15.639346 4508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:17:15.642478 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 4440
I20260811 03:17:15.642683 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-1/wal/instance
I20260811 03:17:15.643285 11504 external_mini_cluster.cc:1428] Setting key c5dea8a1c91fbc1a1d496431a3e08a53
I20260811 03:17:15.648697 4508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:17:15 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418235, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:17:15.677197 4510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.652134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54326 (local address 127.11.60.62:40183)
0811 03:17:15.659616 (+ 7482us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:15.659625 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:15.659652 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:17:15.661812 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:15.661819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:15.661916 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:15.662042 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:15.663699 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.664485 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.666089 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.666585 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.666720 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:15.670808 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:15.670813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:15.670816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:15.670859 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:15.673613 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:15.674641 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:15.674646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:15.674648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:15.674771 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:15.675309 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:15.675312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:15.675314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:15.675591 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:15.675702 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:15.676724 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0811 03:17:15.676925 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7348,"thread_start_us":7221,"threads_started":1}
I20260811 03:17:15.678768 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.652581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40183 (local address 127.0.0.1:54326)
0811 03:17:15.658885 (+ 6304us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.658934 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:15.661485 (+ 2551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:15.662364 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:15.662376 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:15.662920 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:15.663474 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:15.663492 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.664650 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.664655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:15.665936 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:15.665957 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.666827 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.666834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:15.667064 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:15.667775 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:15.667797 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:15.670585 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:15.673763 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:15.673775 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:15.673794 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:15.674508 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:15.674896 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:15.674902 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:15.674906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:15.675188 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:15.675730 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:15.675741 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:15.676256 (+ 515us) client_negotiation.cc:769] Sending connection context
0811 03:17:15.676591 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 03:17:15.677203 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6022,"thread_start_us":5816,"threads_started":1}
WARNING: no policy specified for kudu/127.11.60.60@KRBTEST.COM; defaulting to no policy
I20260811 03:17:15.680612 4508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.60.60@KRBTEST.COM" created.
Aug 11 03:17:15 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418235, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
I20260811 03:17:15.697983 4513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.681548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.11.60.61:42925)
0811 03:17:15.682347 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:15.682358 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:15.682395 (+ 37us) server_negotiation.cc:407] Connection header received
0811 03:17:15.682451 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:15.682460 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:15.682694 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:15.682858 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:15.683562 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.685284 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.686525 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.687439 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.687584 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:15.690954 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:15.690977 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:15.690997 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:15.691045 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:15.694944 (+ 3899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:15.695622 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:15.695633 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:15.695639 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:15.695790 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:15.696197 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:15.696205 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:15.696211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:15.696654 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:15.696820 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:15.697303 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 03:17:15.697647 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":607,"thread_start_us":457,"threads_started":1}
I20260811 03:17:15.698069 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.681499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:37092)
0811 03:17:15.681712 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.681728 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:15.681809 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:15.682865 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:15.682869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:15.683237 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:15.683384 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:15.683395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.685457 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.685461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:15.686378 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:15.686395 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.687611 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.687616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:15.687751 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:15.688380 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:15.688397 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:15.690735 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:15.695084 (+ 4349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:15.695088 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:15.695091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:15.695492 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:15.695928 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:15.695932 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:15.695934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:15.696075 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:15.696784 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:15.696790 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:15.696988 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 03:17:15.697159 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 03:17:15.697526 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 03:17:15.700736 4508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:17:15.702122 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.701501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36064)
0811 03:17:15.701745 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.701901 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 03:17:15.702448 4457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
I20260811 03:17:15.704210 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.703766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36070)
0811 03:17:15.703951 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.704064 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 03:17:15.704463 4508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:44551: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.60.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab.
Entry for principal kudu/127.11.60.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab.
I20260811 03:17:15.727595 11504 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
/tmp/dist-test-taskRu2AW0/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.60:44551
--webserver_interface=127.11.60.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab
--principal=kudu/127.11.60.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.60.20:34679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.60.62:40183,127.11.60.61:42925,127.11.60.60:44551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:17:15.753896 4508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } attempt: 1
I20260811 03:17:15.755337 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.754697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36072)
0811 03:17:15.754964 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.755119 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 03:17:15.756397 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.756046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36088)
0811 03:17:15.756206 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.756289 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 03:17:15.756817 4508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:44551: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
I20260811 03:17:15.806522 4433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } attempt: 5
Aug 11 03:17:15 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418235, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
I20260811 03:17:15.822178 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.807409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:37098)
0811 03:17:15.807657 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.807682 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:15.807814 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:15.808222 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:15.808226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:15.808618 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:15.808796 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:15.808805 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.810419 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.810423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:15.811270 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:15.811281 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.812257 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.812262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:15.812446 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:15.813138 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:15.813157 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:15.815294 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:15.819458 (+ 4164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:15.819462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:15.819465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:15.819890 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:15.820382 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:15.820385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:15.820388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:15.820535 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:15.821345 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:15.821350 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:15.821507 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 03:17:15.821670 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 03:17:15.821917 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 03:17:15.822564 4513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.807628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37098 (local address 127.11.60.61:42925)
0811 03:17:15.807848 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:15.807855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:15.807883 (+ 28us) server_negotiation.cc:407] Connection header received
0811 03:17:15.807970 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:15.807975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:15.808069 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:15.808184 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:15.808971 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.810212 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.811431 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:15.812085 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:15.812221 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:15.815505 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:15.815513 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:15.815518 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:15.815570 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:15.819295 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:15.820033 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:15.820042 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:15.820046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:15.820246 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:15.820664 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:15.820672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:15.820677 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:15.821172 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:15.821359 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:15.821854 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 03:17:15.822164 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 03:17:15.823727 4433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:17:15.824949 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.824533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36094)
0811 03:17:15.824703 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.824784 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 03:17:15.825058 4508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } attempt: 2
I20260811 03:17:15.826049 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.825622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36106)
0811 03:17:15.825782 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.825877 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 03:17:15.826119 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.825753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36112)
0811 03:17:15.825897 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.825979 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 03:17:15.826332 4433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:44551: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
I20260811 03:17:15.827148 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.826737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36124)
0811 03:17:15.826901 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.826995 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 03:17:15.827420 4508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:44551: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
I20260811 03:17:15.877592 4433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } attempt: 1
I20260811 03:17:15.878852 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.878356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36140)
0811 03:17:15.878575 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.878677 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 03:17:15.879985 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.879626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36152)
0811 03:17:15.879796 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.879874 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 03:17:15.880218 4433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:44551: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
I20260811 03:17:15.918705 4508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } attempt: 3
I20260811 03:17:15.920102 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.919484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36168)
0811 03:17:15.919752 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.919895 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 03:17:15.921365 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.920864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36182)
0811 03:17:15.921118 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.921218 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260811 03:17:15.921628 4508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:44551: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
I20260811 03:17:15.948472 4433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } attempt: 2
I20260811 03:17:15.949724 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.949178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36192)
0811 03:17:15.949448 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.949549 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 03:17:15.950872 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:15.950557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36198)
0811 03:17:15.950699 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:15.950767 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 03:17:15.951182 4433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:44551: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
I20260811 03:17:16.057442 4433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } attempt: 3
I20260811 03:17:16.058780 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.058263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36214)
0811 03:17:16.058517 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:16.058609 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 03:17:16.059970 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.059553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36224)
0811 03:17:16.059749 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:16.059834 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260811 03:17:16.060235 4433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:44551: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
I20260811 03:17:16.064889 4508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } attempt: 4
I20260811 03:17:16.066083 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.065495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36228)
0811 03:17:16.065756 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:16.065872 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 03:17:16.067293 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.066903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36236)
0811 03:17:16.067069 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:16.067163 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 03:17:16.067570 4508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.60.60:44551: Network error: Client connection negotiation failed: client connection to 127.11.60.60:44551: connect: Connection refused (error 111)
W20260811 03:17:16.091274 4516 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:17:16.091675 4516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:17:16.091811 4516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:17:16.091953 4516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:17:16.099537 4516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:17:16.099668 4516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:17:16.099793 4516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:17:16.099881 4516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:17:16.099992 4516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:17:16.114748 4516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.60.20:34679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.60.62:40183,127.11.60.61:42925,127.11.60.60:44551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/krb5kdc/kudu_127.11.60.60.keytab
--principal=kudu/127.11.60.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.60.60:44551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.60.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 02:07:30 UTC on ccycloud.quasar-qylciq.root.comops.site
build id 4579572
ASAN enabled
I20260811 03:17:16.116079 4516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:17:16.117437 4516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:17:16.129703 4526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:16.129761 4521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:17:16.129786 4523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:17:16.130627 4516 server_base.cc:1034] running on GCE node
I20260811 03:17:16.131441 4516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:17:16.132933 4516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:17:16.134151 4516 hybrid_clock.cc:648] HybridClock initialized: now 1786418236134100 us; error 35 us; skew 500 ppm
I20260811 03:17:16.134519 4516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 03:17:16 dist-test-slave-tm1g krb5kdc[4346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418236, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:17:16.141517 4516 init.cc:377] Logged in from keytab as kudu/127.11.60.60@KRBTEST.COM (short username kudu)
I20260811 03:17:16.143169 4516 webserver.cc:460] Webserver started at http://127.11.60.60:36963/ using document root <none> and password file <none>
I20260811 03:17:16.143908 4516 fs_manager.cc:359] Metadata directory not provided
I20260811 03:17:16.144042 4516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:17:16.144433 4516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:17:16.147662 4516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/data/instance:
uuid: "17a54761b4784361a59ea8b4a757230e"
format_stamp: "Formatted at 2026-08-11 03:17:16 on dist-test-slave-tm1g"
server_key: "f780b1d6dd32280d106c9d7d234c6052"
server_key_iv: "34065570998ab91887f50b305f860ca7"
server_key_version: "encryptionkey@0"
I20260811 03:17:16.148460 4516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/wal/instance:
uuid: "17a54761b4784361a59ea8b4a757230e"
format_stamp: "Formatted at 2026-08-11 03:17:16 on dist-test-slave-tm1g"
server_key: "f780b1d6dd32280d106c9d7d234c6052"
server_key_iv: "34065570998ab91887f50b305f860ca7"
server_key_version: "encryptionkey@0"
I20260811 03:17:16.153743 4516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 03:17:16.157313 4534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:17:16.158855 4516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:17:16.159041 4516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/data,/tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/wal
uuid: "17a54761b4784361a59ea8b4a757230e"
format_stamp: "Formatted at 2026-08-11 03:17:16 on dist-test-slave-tm1g"
server_key: "f780b1d6dd32280d106c9d7d234c6052"
server_key_iv: "34065570998ab91887f50b305f860ca7"
server_key_version: "encryptionkey@0"
I20260811 03:17:16.159261 4516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:17:16.187011 4516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:17:16.191430 4516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:17:16.191726 4516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:17:16.211069 4516 rpc_server.cc:307] RPC server started. Bound to: 127.11.60.60:44551
I20260811 03:17:16.211112 4586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.60.60:44551 every 8 connection(s)
I20260811 03:17:16.212482 4516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/data/info.pb
I20260811 03:17:16.215155 11504 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu as pid 4516
I20260811 03:17:16.215356 11504 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRu2AW0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417858274425-11504-0/minicluster-data/master-2/wal/instance
I20260811 03:17:16.215814 11504 external_mini_cluster.cc:1428] Setting key ddaa9bfcf71802273a46b75709664a78
I20260811 03:17:16.216511 4587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:17:16.226670 4587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:17:16 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
I20260811 03:17:16.237465 4433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } attempt: 4
Aug 11 03:17:16 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418236, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
I20260811 03:17:16.241523 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.217709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.11.60.60:44551)
0811 03:17:16.218563 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.218580 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.218630 (+ 50us) server_negotiation.cc:407] Connection header received
0811 03:17:16.220294 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.220322 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.220837 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.221298 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.222703 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.224769 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.226140 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.227252 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.227459 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.232503 (+ 5044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.232510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.232519 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.232556 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.236223 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.237048 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.237059 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.237068 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.237271 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.237753 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.237759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.237762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.238555 (+ 793us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.238900 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.239583 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.239998 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":339,"threads_started":1}
Aug 11 03:17:16 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418235, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.62@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
I20260811 03:17:16.247318 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.229602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.62:40183 (local address 127.0.0.1:54332)
0811 03:17:16.230565 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:16.230608 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:16.230806 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:16.231244 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:16.231252 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:16.231934 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:16.232177 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:16.232186 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.233299 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.233303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:16.235199 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:16.235220 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.235996 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.236004 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:16.236202 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.236874 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:16.236897 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:16.239786 (+ 2889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:16.243899 (+ 4113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:16.243914 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:16.243922 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:16.244551 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:16.245058 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:16.245065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:16.245071 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:16.245262 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:16.246090 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:16.246098 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:16.246292 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 03:17:16.246512 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 03:17:16.246837 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":565,"threads_started":1}
I20260811 03:17:16.247344 4591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.229637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54332 (local address 127.11.60.62:40183)
0811 03:17:16.230225 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.230233 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.230709 (+ 476us) server_negotiation.cc:407] Connection header received
0811 03:17:16.230972 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.230976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.231042 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.231134 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.232343 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.233107 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.235366 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.235802 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.235899 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.240160 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.240166 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.240170 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.240211 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.243767 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.244716 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.244723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.244728 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.244926 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.245507 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.245514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.245523 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.245909 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.246082 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.246678 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.246922 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":275,"threads_started":1}
I20260811 03:17:16.249173 4587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:17:16.255528 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.238134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36254)
0811 03:17:16.238454 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:16.238472 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:16.238579 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:16.239538 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:16.239543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:16.240076 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:16.240306 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:16.240318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.241950 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.241957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:16.243278 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:16.243292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.244049 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.244056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:16.244261 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.244946 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:16.244974 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:16.248719 (+ 3745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:16.252951 (+ 4232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:16.252957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:16.252961 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:16.253337 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:16.253741 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:16.253747 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:16.253750 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:16.253894 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:16.254520 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:16.254525 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:16.254687 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 03:17:16.254828 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 03:17:16.255142 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"spinlock_wait_cycles":63360}
I20260811 03:17:16.255554 4593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.238223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.11.60.60:44551)
0811 03:17:16.239189 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.239197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.239227 (+ 30us) server_negotiation.cc:407] Connection header received
0811 03:17:16.239276 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.239281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.239353 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.240108 (+ 755us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.240474 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.241536 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.243426 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.243890 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.244049 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.248985 (+ 4936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.248992 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.248996 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.249030 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.252819 (+ 3789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.253467 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.253473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.253476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.253620 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.254023 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.254028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.254031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.254362 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.254561 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.254944 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.255155 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":566,"threads_started":1}
Aug 11 03:17:16 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.62@KRBTEST.COM
Aug 11 03:17:16 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418236, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
I20260811 03:17:16.261325 4433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670: Bootstrap starting.
I20260811 03:17:16.264078 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.245533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54340 (local address 127.11.60.62:40183)
0811 03:17:16.246241 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.246250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.246274 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:17:16.246362 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.246368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.246438 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.247791 (+ 1353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.248373 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.249353 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.251170 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.251604 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.251742 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.257204 (+ 5462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.257209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.257212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.257248 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.260858 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.261542 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.261547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.261551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.261743 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.262141 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.262148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.262151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.262639 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.263167 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.263292 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.263652 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":350,"threads_started":1}
I20260811 03:17:16.266172 4433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670: Neither blocks nor log segments found. Creating new log.
I20260811 03:17:16.267673 4433 log.cc:824] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670: Log is configured to *not* fsync() on all Append() calls
I20260811 03:17:16.268200 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.250046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.61:42925 (local address 127.0.0.1:37106)
0811 03:17:16.250482 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:16.250505 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:16.250618 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:16.251226 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:16.251233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:16.251836 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:16.252051 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:16.252063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.253537 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.253544 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:16.255152 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:16.255167 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.256393 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.256406 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:16.256641 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.257835 (+ 1194us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:16.257852 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:16.260515 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:16.265219 (+ 4704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:16.265224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:16.265227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:16.265769 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:16.266403 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:16.266407 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:16.266412 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:16.266596 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:16.267280 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:16.267285 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:16.267447 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 03:17:16.267583 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 03:17:16.267861 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317}
I20260811 03:17:16.268666 4513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.250449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37106 (local address 127.11.60.61:42925)
0811 03:17:16.250625 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.250631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.250650 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:17:16.250714 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.250721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.250796 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.250901 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.252232 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.253322 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.255342 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.256021 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.256125 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.260734 (+ 4609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.260740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.260745 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.260788 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.265021 (+ 4233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.265929 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.265937 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.265943 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.266190 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.266729 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.266736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.266742 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.267160 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.267313 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.267702 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.268004 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 03:17:16.269855 4587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:17:16.273424 4433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670: No bootstrap required, opened a new log
Aug 11 03:17:16 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.60.61@KRBTEST.COM
Aug 11 03:17:16 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418236, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.60@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
I20260811 03:17:16.283963 4433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } }
I20260811 03:17:16.284325 4433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 027b881a8617460f951497682be3d670, State: Initialized, Role: FOLLOWER
I20260811 03:17:16.284416 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.265729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37116 (local address 127.11.60.61:42925)
0811 03:17:16.266393 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.266401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.266427 (+ 26us) server_negotiation.cc:407] Connection header received
0811 03:17:16.266605 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.266610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.266709 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.266812 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.268585 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.270036 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.271677 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.272121 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.272259 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.277294 (+ 5035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.277299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.277302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.277340 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.280755 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.281608 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.281614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.281617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.281755 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.282161 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.282170 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.282172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.282503 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.282772 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.283684 (+ 912us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.283934 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":291,"threads_started":1}
I20260811 03:17:16.284869 4433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } }
I20260811 03:17:16.285795 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.270767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36258)
0811 03:17:16.270922 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:16.270942 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:16.271033 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:16.271412 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:16.271416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:16.271891 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:16.272075 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:16.272086 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.273372 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.273377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:16.274740 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:16.274757 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.275825 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.275834 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:16.276031 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.276613 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:16.276635 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:16.279216 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:16.282978 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:16.282983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:16.282986 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:16.283504 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:16.283922 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:16.283926 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:16.283929 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:16.284073 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:16.284730 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:16.284734 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:16.284896 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 03:17:16.285075 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 03:17:16.285397 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 03:17:16.285866 4593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.271023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.11.60.60:44551)
0811 03:17:16.271168 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.271173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.271187 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:17:16.271229 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.271232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.271292 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.271395 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.272237 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.273226 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.274926 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.275639 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.275787 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.279425 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.279432 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.279437 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.279475 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.282838 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.283637 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.283642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.283645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.283785 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.284189 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.284194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.284198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.284538 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.284679 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.285160 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.285548 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":162,"server-negotiator.queue_time_us":53}
I20260811 03:17:16.288458 4433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:17:16.291145 4597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } } }
I20260811 03:17:16.291632 4597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:16.292172 4587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e: Bootstrap starting.
I20260811 03:17:16.293376 11504 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:17:16.295976 4406 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "027b881a8617460f951497682be3d670"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54340
I20260811 03:17:16.296407 4406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 03:17:16.296608 4406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:17:16.297969 4587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e: Neither blocks nor log segments found. Creating new log.
I20260811 03:17:16.299661 4587 log.cc:824] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e: Log is configured to *not* fsync() on all Append() calls
I20260811 03:17:16.300271 4406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } }
I20260811 03:17:16.301836 4406 leader_election.cc:288] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [CANDIDATE]: Term 1 election: Requested vote from peers 6614417d01f54471aa6bc38c61328923 (127.11.60.61:42925), 17a54761b4784361a59ea8b4a757230e (127.11.60.60:44551)
I20260811 03:17:16.304545 4587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e: No bootstrap required, opened a new log
I20260811 03:17:16.316282 4587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } }
I20260811 03:17:16.316753 4587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a54761b4784361a59ea8b4a757230e, State: Initialized, Role: FOLLOWER
I20260811 03:17:16.317282 4587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } }
W20260811 03:17:16.321298 4610 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:17:16.321561 4610 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:17:16.324067 4614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } } }
I20260811 03:17:16.324744 4614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:16.327250 4587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:17:16.327292 4562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "027b881a8617460f951497682be3d670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "17a54761b4784361a59ea8b4a757230e"
I20260811 03:17:16.327854 4562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:17:16.332274 4562 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 027b881a8617460f951497682be3d670 in term 1.
I20260811 03:17:16.333216 4378 leader_election.cc:302] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 027b881a8617460f951497682be3d670, 17a54761b4784361a59ea8b4a757230e; no voters:
I20260811 03:17:16.333793 4597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 03:17:16.335799 4383 tablet.cc:2369] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:17:16.338588 4597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [term 1 LEADER]: Becoming Leader. State: Replica: 027b881a8617460f951497682be3d670, State: Running, Role: LEADER
I20260811 03:17:16.339213 4597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } }
I20260811 03:17:16.311882 4361 master_runner.cc:186] Error getting master registration for 127.11.60.61:42925: OK, instance_id { permanent_uuid: "6614417d01f54471aa6bc38c61328923" instance_seqno: 1786418235628210 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:17:16.345827 4361 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 03:17:16.349130 4617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 027b881a8617460f951497682be3d670. Latest consensus state: current_term: 1 leader_uuid: "027b881a8617460f951497682be3d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } } }
I20260811 03:17:16.349589 4617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:16.350970 4620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:17:16.362169 4620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:17:16.366786 4508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } attempt: 5
Aug 11 03:17:16 dist-test-slave-tm1g krb5kdc[4346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786418235, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.60.61@KRBTEST.COM for kudu/127.11.60.60@KRBTEST.COM
I20260811 03:17:16.386219 4562 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [term 1 FOLLOWER]: Refusing update from remote peer 027b881a8617460f951497682be3d670: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:17:16.387605 4617 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:17:16.399330 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.367508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.60.60:44551 (local address 127.0.0.1:36266)
0811 03:17:16.368023 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 03:17:16.368049 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 03:17:16.368202 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:17:16.369387 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:17:16.369395 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:17:16.370260 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:17:16.370577 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:16.370595 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.372569 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.372578 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:16.374660 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:17:16.374681 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.376344 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.376354 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:17:16.376658 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.377548 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:17:16.377583 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:17:16.383696 (+ 6113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:17:16.391064 (+ 7368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:16.391072 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:16.391080 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:16.391852 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:16.394041 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:17:16.394048 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:17:16.394053 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:17:16.394272 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:17:16.395983 (+ 1711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:17:16.395991 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:17:16.396221 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 03:17:16.396418 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 03:17:16.398768 (+ 2350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335}
I20260811 03:17:16.399950 4593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.368467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.11.60.60:44551)
0811 03:17:16.368696 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.368704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.368730 (+ 26us) server_negotiation.cc:407] Connection header received
0811 03:17:16.368879 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.368888 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.369005 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.369133 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.370828 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.372355 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.374908 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.375862 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.376025 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.385143 (+ 9118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.385153 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.385161 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.385229 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.390735 (+ 5506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.392149 (+ 1414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.392160 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.392167 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.392389 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.394966 (+ 2577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.394974 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.394981 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.395518 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.395790 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.396926 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.397632 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 03:17:16.407258 4508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923: Bootstrap starting.
I20260811 03:17:16.412168 4508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923: Neither blocks nor log segments found. Creating new log.
I20260811 03:17:16.416117 4508 log.cc:824] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923: Log is configured to *not* fsync() on all Append() calls
I20260811 03:17:16.423100 4508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923: No bootstrap required, opened a new log
I20260811 03:17:16.437268 4508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } }
I20260811 03:17:16.437747 4508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6614417d01f54471aa6bc38c61328923, State: Initialized, Role: FOLLOWER
I20260811 03:17:16.438346 4508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } }
W20260811 03:17:16.443996 4540 tablet.cc:2369] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:17:16.448603 4614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 027b881a8617460f951497682be3d670. Latest consensus state: current_term: 1 leader_uuid: "027b881a8617460f951497682be3d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } } }
I20260811 03:17:16.448926 4614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:16.453487 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.364826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54348 (local address 127.11.60.62:40183)
0811 03:17:16.365094 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.365103 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.410573 (+ 45470us) server_negotiation.cc:407] Connection header received
0811 03:17:16.411179 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.411185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.411280 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.411362 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.414467 (+ 3105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.415622 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.422791 (+ 7169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.423544 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.423675 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.432924 (+ 9249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.432932 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.432936 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.433011 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.437068 (+ 4057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.441864 (+ 4796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.441876 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.441882 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.442232 (+ 350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.448725 (+ 6493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.448735 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.448741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.449474 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.449828 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.452556 (+ 2728us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.452944 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 03:17:16.462667 4627 mvcc.cc:205] Tried to move back new op lower bound from 7317169096219713536 to 7317169096053088256. Current Snapshot: MvccSnapshot[applied={T|T < 7317169096219713536}]
I20260811 03:17:16.462080 4633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } } }
I20260811 03:17:16.464005 4633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:16.464411 4508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:17:16.464463 4483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "027b881a8617460f951497682be3d670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6614417d01f54471aa6bc38c61328923"
I20260811 03:17:16.464962 4483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:17:16.469173 4483 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 027b881a8617460f951497682be3d670 in term 1.
I20260811 03:17:16.470783 4482 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [term 1 FOLLOWER]: Refusing update from remote peer 027b881a8617460f951497682be3d670: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:17:16.473266 4617 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:17:16.495026 4614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "027b881a8617460f951497682be3d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } } }
I20260811 03:17:16.495337 4614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:16.502590 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.394349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37126 (local address 127.11.60.61:42925)
0811 03:17:16.394583 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.394589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.415064 (+ 20475us) server_negotiation.cc:407] Connection header received
0811 03:17:16.416749 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.416757 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.416843 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.416910 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.425655 (+ 8745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.430505 (+ 4850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.450312 (+ 19807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.451061 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.451169 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.490626 (+ 39457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.490639 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.490646 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.490707 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.496182 (+ 5475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.497820 (+ 1638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.497825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.497828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.498256 (+ 428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.498935 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.498940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.498943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.499352 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.499609 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.501953 (+ 2344us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.502187 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 03:17:16.503304 4620 catalog_manager.cc:1347] Generated new cluster ID: 9edb253c52024c61a43a1943c9525ad1
I20260811 03:17:16.503433 4620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:17:16.495926 4655 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 03:17:16.505529 4617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "027b881a8617460f951497682be3d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } } }
I20260811 03:17:16.505828 4617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:16.507196 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:16.394360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.11.60.60:44551)
0811 03:17:16.394542 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.394547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.415735 (+ 21188us) server_negotiation.cc:407] Connection header received
0811 03:17:16.416272 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.416278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.416370 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.416448 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.441426 (+ 24978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.442588 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.456617 (+ 14029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:16.457303 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:16.457408 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:16.495767 (+ 38359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:17:16.495773 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:17:16.495777 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:17:16.495821 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:17:16.501707 (+ 5886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.502884 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.502893 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.502899 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.503088 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:17:16.503606 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:17:16.503611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:17:16.503614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:17:16.504031 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:17:16.504288 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:16.504709 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 03:17:16.504945 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 03:17:16.510931 4516 master_runner.cc:186] Error getting master registration for 127.11.60.61:42925: OK, instance_id { permanent_uuid: "6614417d01f54471aa6bc38c61328923" instance_seqno: 1786418235628210 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:17:16.523851 4516 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 03:17:16.525303 4633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 027b881a8617460f951497682be3d670. Latest consensus state: current_term: 1 leader_uuid: "027b881a8617460f951497682be3d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } } }
I20260811 03:17:16.525663 4633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:16.526490 4656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "027b881a8617460f951497682be3d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } } }
I20260811 03:17:16.526952 4656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:17:16.535025 4657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "027b881a8617460f951497682be3d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "027b881a8617460f951497682be3d670" member_type: VOTER last_known_addr { host: "127.11.60.62" port: 40183 } } peers { permanent_uuid: "6614417d01f54471aa6bc38c61328923" member_type: VOTER last_known_addr { host: "127.11.60.61" port: 42925 } } peers { permanent_uuid: "17a54761b4784361a59ea8b4a757230e" member_type: VOTER last_known_addr { host: "127.11.60.60" port: 44551 } } }
I20260811 03:17:16.535544 4657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670 [sys.catalog]: This master's current role is: LEADER
I20260811 03:17:16.539167 4620 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:17:16.539819 4660 catalog_manager.cc:1259] Loaded cluster ID: 9edb253c52024c61a43a1943c9525ad1
I20260811 03:17:16.539918 4660 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923: loading cluster ID for follower catalog manager: success
I20260811 03:17:16.541030 4620 catalog_manager.cc:1504] Loading token signing keys...
W20260811 03:17:16.541095 4660 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 03:17:16.541214 4660 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:17:16.559458 4620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 027b881a8617460f951497682be3d670: Generated new TSK 0
I20260811 03:17:16.560401 4620 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 03:17:16.654137 4594 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.640195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54364 (local address 127.11.60.62:40183)
0811 03:17:16.640460 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.640469 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.650999 (+ 10530us) server_negotiation.cc:407] Connection header received
0811 03:17:16.651386 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.651411 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.651531 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.651625 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.653749 (+ 2124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54364: BlockingRecv error: recv got EOF from 127.0.0.1:54364 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260811 03:17:16.658316 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.652285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.11.60.61:42925)
0811 03:17:16.652484 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.652492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.655720 (+ 3228us) server_negotiation.cc:407] Connection header received
0811 03:17:16.656257 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.656266 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.656399 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.656494 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.657874 (+ 1380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37130: BlockingRecv error: recv got EOF from 127.0.0.1:37130 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 03:17:16.661001 4590 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.646533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.11.60.60:44551)
0811 03:17:16.646854 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.646863 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.655200 (+ 8337us) server_negotiation.cc:407] Connection header received
0811 03:17:16.658952 (+ 3752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.658962 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.659078 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.659173 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.660635 (+ 1462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36276: BlockingRecv error: recv got EOF from 127.0.0.1:36276 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
W20260811 03:17:16.688670 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.686331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.11.60.61:42925)
0811 03:17:16.686579 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.686588 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.687148 (+ 560us) server_negotiation.cc:407] Connection header received
0811 03:17:16.687407 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.687417 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.687536 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.687665 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.688442 (+ 777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37144: BlockingRecv error: recv got EOF from 127.0.0.1:37144 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 03:17:16.696599 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.692802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.11.60.61:42925)
0811 03:17:16.693058 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.693067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.694840 (+ 1773us) server_negotiation.cc:407] Connection header received
0811 03:17:16.695164 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.695169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.695249 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.695319 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.696327 (+ 1008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37156: BlockingRecv error: recv got EOF from 127.0.0.1:37156 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 03:17:16.708925 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.706647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37160 (local address 127.11.60.61:42925)
0811 03:17:16.706884 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.706893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.707510 (+ 617us) server_negotiation.cc:407] Connection header received
0811 03:17:16.707745 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.707754 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.707838 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.707963 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.708710 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37160: BlockingRecv error: recv got EOF from 127.0.0.1:37160 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 03:17:16.719499 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.717137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.11.60.61:42925)
0811 03:17:16.717385 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.717394 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.717954 (+ 560us) server_negotiation.cc:407] Connection header received
0811 03:17:16.718196 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.718201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.718286 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.718466 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.719245 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37172: BlockingRecv error: recv got EOF from 127.0.0.1:37172 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 03:17:16.733253 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.730964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.11.60.61:42925)
0811 03:17:16.731225 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.731234 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.731793 (+ 559us) server_negotiation.cc:407] Connection header received
0811 03:17:16.732029 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.732033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.732127 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.732239 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.733021 (+ 782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37174: BlockingRecv error: recv got EOF from 127.0.0.1:37174 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260811 03:17:16.755229 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.752715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.11.60.61:42925)
0811 03:17:16.752961 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.752970 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.753590 (+ 620us) server_negotiation.cc:407] Connection header received
0811 03:17:16.753821 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.753825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.753903 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.754014 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.754902 (+ 888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37190: BlockingRecv error: recv got EOF from 127.0.0.1:37190 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 03:17:16.793146 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.790743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37194 (local address 127.11.60.61:42925)
0811 03:17:16.791003 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.791013 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.791592 (+ 579us) server_negotiation.cc:407] Connection header received
0811 03:17:16.791848 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.791853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.791930 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.792052 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.792890 (+ 838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37194: BlockingRecv error: recv got EOF from 127.0.0.1:37194 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 03:17:16.868228 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:16.865690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.11.60.61:42925)
0811 03:17:16.865878 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:16.865885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:16.866492 (+ 607us) server_negotiation.cc:407] Connection header received
0811 03:17:16.866731 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:16.866739 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:16.866869 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:16.866991 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:16.867983 (+ 992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37196: BlockingRecv error: recv got EOF from 127.0.0.1:37196 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 03:17:17.010160 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:17.007880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37204 (local address 127.11.60.61:42925)
0811 03:17:17.008146 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:17.008153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:17.008565 (+ 412us) server_negotiation.cc:407] Connection header received
0811 03:17:17.008801 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:17.008805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:17.008880 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:17.008963 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:17.009909 (+ 946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37204: BlockingRecv error: recv got EOF from 127.0.0.1:37204 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 03:17:17.272594 4595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:17:17.270603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.11.60.61:42925)
0811 03:17:17.270868 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:17.270877 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:17.271130 (+ 253us) server_negotiation.cc:407] Connection header received
0811 03:17:17.271399 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:17.271403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:17.271480 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:17.271613 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:17:17.272379 (+ 766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37208: BlockingRecv error: recv got EOF from 127.0.0.1:37208 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 03:17:17.510133 4655 catalog_manager.cc:1259] Loaded cluster ID: 9edb253c52024c61a43a1943c9525ad1
I20260811 03:17:17.510257 4655 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e: loading cluster ID for follower catalog manager: success
I20260811 03:17:17.512974 4655 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e: acquiring CA information for follower catalog manager: success
I20260811 03:17:17.514914 4655 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 17a54761b4784361a59ea8b4a757230e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:17:17.544524 4660 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923: acquiring CA information for follower catalog manager: success
I20260811 03:17:17.546944 4660 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6614417d01f54471aa6bc38c61328923: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:17:17.795466 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:17.789018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37222 (local address 127.11.60.61:42925)
0811 03:17:17.789860 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:17.789868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:17.789890 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:17:17.789989 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:17.789995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:17.790178 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:17.790351 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:17:17.790805 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:17.791553 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:17.792948 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:17.793356 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:17.793498 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:17.794038 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:17:17.794714 (+ 676us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:17:17.794876 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:17.794978 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 03:17:17.795065 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":550,"threads_started":1}
I20260811 03:17:17.810963 4742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:17.802778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36290 (local address 127.11.60.60:44551)
0811 03:17:17.803419 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:17.803425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:17.803760 (+ 335us) server_negotiation.cc:407] Connection header received
0811 03:17:17.804005 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:17.804009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:17.804181 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:17.804354 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:17:17.804971 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:17.805962 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:17.808047 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:17.808583 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:17.808694 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:17.809414 (+ 720us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:17:17.810080 (+ 666us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:17:17.810213 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:17.810440 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 03:17:17.810520 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":296,"threads_started":1}
I20260811 03:17:17.841344 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:17.818547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37236 (local address 127.11.60.61:42925)
0811 03:17:17.818793 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:17.818803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:17.834793 (+ 15990us) server_negotiation.cc:407] Connection header received
0811 03:17:17.834949 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:17.834955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:17.835052 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:17.835147 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:17:17.835895 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:17.836996 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:17.839093 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:17.839739 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:17.839878 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:17.840521 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:17:17.840667 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:17:17.840728 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:17.840954 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 03:17:17.841043 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 03:17:17.841596 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:17.818986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54374 (local address 127.11.60.62:40183)
0811 03:17:17.832061 (+ 13075us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:17.832071 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:17.832100 (+ 29us) server_negotiation.cc:407] Connection header received
0811 03:17:17.832266 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:17.832274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:17.832561 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:17.832808 (+ 247us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:17:17.834587 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:17.835706 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:17.837902 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:17.838609 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:17.838742 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:17.840067 (+ 1325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:17:17.840570 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:17:17.840686 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:17.841195 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 03:17:17.841290 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12907,"thread_start_us":12783,"threads_started":1}
I20260811 03:17:17.843422 4742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:17:17.820563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.11.60.60:44551)
0811 03:17:17.820779 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 03:17:17.820788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:17:17.834463 (+ 13675us) server_negotiation.cc:407] Connection header received
0811 03:17:17.835164 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:17:17.835170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:17:17.835273 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:17:17.835356 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:17:17.836291 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:17.837515 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:17.840689 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:17:17.841403 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:17:17.841510 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:17:17.842413 (+ 903us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:17:17.842546 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:17:17.842605 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 03:17:17.843008 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 03:17:17.843107 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 03:17:17.846904 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 4361
I20260811 03:17:17.847986 4428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:17:17.963887 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 4361
I20260811 03:17:17.989485 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 4440
I20260811 03:17:17.990697 4503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:17:18.101083 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 4440
I20260811 03:17:18.125458 11504 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu pid 4516
I20260811 03:17:18.126706 4582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 03:17:18.234454 11504 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRu2AW0/build/asan/bin/kudu with pid 4516
2026-08-11T03:17:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5147 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10202 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (379920 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST